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INTRODUCTION 
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This volume is one of a series th^at documents Stage Three of a project on 
Quality in the Pell Grant Delivery System. Other volumes concern the substantive 
findings of the project and recommendations for actions to correct the problems 
' found. . V 

In September, 1980, the Of f ice of Student Financial Assistance (OSFA) of the 
U.S. Department of Education contracted with Advanced Technology, Inq., of McLean 
and Reston, Virginia, to conduct a three-year study to ^assess the accuracy and 
reliability of the Basic Educational Opportunity Grant (BEOG) Program and 
recommend administrative changes to improve it. Westat, Inc., of Rockville, 
Maryland, has served as a subcontractor to Advanced Technology throughout the study. 
In 1981, the name of the program was changed to the Pell Grant Program. 

^ Pell Grants are entitlements available to students to support part of the cost of 
pursuing postsecondary education. The students, the schools they attend, and the 
programs in which they enroll must satisfy categorical eligiBility requirements. Once 
these requirements have been met, the amount of the student's grant depends on the 
student's "need," which is a. function of both the cdst of the program and of the 
student's ability to pay that cost from student and family assets and income. The 
calculation oi costs and the student/family contribution are prescribed by program 
formulae. Awards can be made in error if the student or the program is deemed to 
. satisfy categorical criteria which they do not, in fact, meet, or if the student's need is 
, miscalculated, or if the institution miscalculates the student!? costs, enrollment 
status, or award.' The money for awards is usually allocated from the U.S. Treasury to 
the institutions,^hich payv4t to individual students or credit it t6 their accounts.\^ 

During Stage One of the study— the first year of the contract (1980-81)— 
Advanced Technology and Westat examined a national sample of Pell Grant recipients 
to deter mine^ligibility and award calculation error. Westat drew the national sample 

s 
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of ^,500 recipients and interviewed them and their parents about their eligibility and 
financial situation, examining documentation at the same time. Advanced Technology 
hired staff to visit the institutions attended by ..these students, examine the records on 
the students and any supporting documents on file, and interview the financial aid 
administrators. This data collection procedure was also followed in Stage Three— the 
third year of the contract (1982-83)--and is described below. 

• > . 

The Stage One study determined program-wide rates of discrepancy between 
actual grant awards and the awards that should have been made according to program • 
rules and the documents examined, and attributed these discrepancies to institutions, 
recipients or their parents, and application processors. On the basis of these 
discrepancy rates, tl^ study also identified error-prone groups of recipients. Finally, 
S,tage One suggested feasible corrective management activities to reduce error rates 
for every area in which error rates were excessive. - ^ 

During Stage Two (October, 1981, to December, 1982), Advanced Technology 
began the design of a quality control system for the iPell Program and made some error 
analyses and corrective action recommendations for specific features -of related 
studpnt aid programs. The Department of Education began installing coKective 
measures, including a requirement for greatly increased validation of Pell Grant 
applicants on a limited number of application items, rather than a small sample on 
more items. In^ 1983, the quality control system design component became a separate 
project. 

Stage Three of the study (1982-83) has essentially been a replication of Stage 
One, with the objective of determining changes in program error oyer time, especially 
changes potentially brought about by the extended validation requirement. As in Stage • 
One, Westaf developed a sampling procedure and interviev/ed the sampled parents and 
students, and Advanced Technology visited institutions to examine documents \n 
* student files and interview financial aid administrators. During Stage Three Advanced 
Technology made a prelifmnary visit to each of 3^7 institutions to draw the sample ofv 
^,109 students on site (during St^ge One Westat had drawn the sample at its Home 
office from lists supplied by the institutions) and to gather data for assessment of 
compliance with the new validation requirement. ^ 
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A findings report (Volume 1) and corrective action- recommendations based on 
these data (Volume 2) aire again accompanying this report on methodology (Volume 3). 
The contents of this volume consist of the procedures and methods of the study 
including sample selection, institutional visits, student and parent data collection, data 
processing, and data analysis. In addition, sampling response rates and detailed tables 
of the \^ariance estimates for Volume 1 are included in a final chapter. Methodology 
that is deemed to be essential for understanding the findings of the report (Valume 1) 
is included in that volume. Thus," Volume 1 includes error definitions, strength of 
documentation, weighting and nonresponse adjustment, and experimental bias 
assessment in addition to findings on error'in the Pell Grant prograrrv. 
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2 

i^AMPLE SELECTION 



. This chapter outlines the procedures used to select the sample of institutions for 
inclusion in the study and to select students at those institutions for interviewing. 
Two sampling t^hniques were used to select' the students to be interviewed: one for 
students attending* institutions distributing Pell Grants und^r the Regular 
Disbursement System (RDS), and one for students attending institutions using the 
Alternate Disbursement System (ADS).* The result was a seli-weighting probability 
sample of all Pell recipients. 

2.1 OVERVIEW OF SAMPLING FOR THE STUPY \ 

*2ll.l The RDS Sample . 

The^sample of students attending the RDS institutions was selected using a two- 
stage s/mpling procedure. The institutions were arranged in geographic clusters 
during/the' first stage of selection, with stratification iy type of institution (less than 
two years', two to three years, and four years or more) and type ol control (public, 
private, ai^d prDpYietaryJ done at the second stage. The sampling frame for the RDS 
institutions was constructed from the Pell Grant Program Institution Universe File and 
the October 1981 PD&gf€;S5jRepprt Error/Unreasonable File. Only active institutions 
within the contiguous ^8 states were included. 

Westat selected institutions according to the foUowing^teps: 



• They ordered tn<e file by geographic code, with clusters formed consisting 
bfjcon^ecutiye schools. Each cluster contained about eight schools. 



* Additional detail on the sample design can be found in "Technical Report on Stage 
Three Sampling Prbcedufes for Validation Evaluation Procedu^es," December, 1982. 



/ 



4 



r 



• Clusters were sampled with probability proportional to total recipients in 
the cluster. 

• They assigned an adjusted measure of size (AMOS) to each school within 
the sampled clusters, where: 

AMOS,. = IMOSj/CM'OSj ^ . . ' 

,i = school 

j = cluster ' ^ , " 

IMOS = the institution measure of size 

CMOS = the cluster measure of 6ize , ' 

• They ordered schools in sample dusters, ignoring the cluster identifier, by 
^ the following type-control strata: 

School control (public, private, proprietary) . 

• . ♦ 

i' - School type (offering 1 to 2 year programs, offering 2 to 3 year 
programs, offering ^ or more year programs) • 

• They sampled schools systematically* from the Ordered file with probability 
proportional to "the adjusted measure of size (AMOS). 

^ . ' : * ' ^ 

The student sample was drawn on sit^t each institution by data collectors hired 
and trained by Advanced Technology. At each institution, the data coUectoj^ obtained 
a "list" of Pell Grant recipient^from the institution and numbered them from 1 to n. 
Westat provided for each institution a list of the sequence numbers to be sampled, 
based on a fixed interval with random start. The intervals were c^ul^rted to give 
each student an equal probability dt selection, and were based on the number of 
expected recipient's. Sometimes the "list" consisted of Student Aid Reports (SAR) in a 
pile or file folder, the financial aid files of all the students at the institution, or other 
unusual record formats. 

Data collectors occasionally found contaminated lists which included either 
recipients of other t^^pes of aid or people who had not, in fact, registered and received 
a Pell award at the institution. Where it was practical to do so, the data collector 
cleaned the list before selecting the sample. Where this was not practical, the list was 
used in its original form, resulting in oversampling. Checking files or records of the 
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sample enabled the data collector to eliminate from this oversized satnple ineligibles 
who were not part of the study population. Any ineligibles still remaining in the 
sample were screened out prior to the student interview or the student record 
\ abstract. 

2.1.2 The ADS Sample 

Students .attending institutions using the Alternate Disbursment System were 
selected from a back-up tape used by the Treasury to mail checks to Pell Grant 
recipients at ADS institutions. The ADS sample was not clustered by institution for 
two reasons. First, there was no easily available indicator of what school these 
Students attend. Second, there are very few students in the ADS population compared 
to the RDS students; therefore, we expected a "^'proportional samplel of only about '20 
students, minimizing any gains that might have been acquired in the conduct of the 
survey through clustering. The ADS student sample was selected according to the 
following steps: 

• The Treasury tape was purged of duplicate records. 

• The number of unique records in unduplicated file was determined. 



• A simple random sample of students was selected from the unduplicated 
file. 



The ADS sample was selected using the same selection probabilities as the RDS 
sample, resulting in a self-weighting probability sample combining both categories of 
students. 

2.1.3 The Control Group Samples 

To provide a means of determining whether experimental bias existed, we 
selected two control groups, one from the sampled institutions (Institution Control 

Group or ICG) and one from both sampled and non-sampled institutions using the 

7 

Computed Applicant Record (from the central processor) Control Group (jpARCG). 
The ICG was selected from the sample lists used to draw the RDS' sample. A fixed 
interval five times greater than the main sample interval was used with a new random 
start, resulting in a sample of about 800 students. Thus, institutions with a small 
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\ number of sampled students frequently had no control sample students'. For 12 
institutions, no list or reasonable facsimile was available and no control sample could 
be drawn. * " t 



The CARCG was drawn from the central processor's applicant file of more than 
seven million records. A sample 0^^20,000 applicant records was selected using a fixed 
interva^l and a random start. This sample was matched with the Pell Recipient History 
File to exclude non-recipients and separate students attending sampled and non- 
sampled institutions. The resulting file contained 9,051 recipients at non-sampled 
institutions and 1,997 recipients at sampled institutions. 

2.2 DETAILED SAMPLING PLAN 

« 

The primary objective of the sample design was to choose a probability sample of 
students enrolled at educational institutions participating in the 1982-83 Pell Grant 
program. The statistically representative sample was designed to minimize the 
variability of characteristics among possible samples that can result from sample 
selection. The method used to achieve this goal was sample selection with probability 
proportional to a measure of size (MOS) that is presumed to be correlated with the 
statistics being estimated. In the institution sample selection, the measure of size 
used was the number of Pell Grant recipients reported by the institution in the 
October 1981 Progress Report. The October )981 Progress Report was used for two 
reasons: (1) the October report satisfied the need for a sample of students receiving 
grants in the Fall; and (2) current year (1982-83) Progress Reports were incomplete. 

2.2.1 Sample Size and Precision 

The choice of sample size was determined, in large part, by the budget and time 
contraints of the study and the level of precision sought for sample estimates. The 
maximum number of institutions that reasonably could be contacted during the period 
provided for institutional data collection was about 300. In terms of the recipient 
sample, the budget permitted 3,600 completed interviews. 

Based on the data available from the 1980-81 study (Stage One), . basic 
assumptions on parameters that were not available at that time, we estimated the 
precision that can be expected with 300 institutions and an average of 12 students per 

2-* 
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institution. . A student characteristic such as absolute dollar error (3c) canvbejooked at 
as the product of two variables to be estimated from the survey: the average absolute 
dollar value of the error given that an error was made and thej^ropprtion of cases 
in error (P); thus 

• • 

x'= Pe 

The relvariance (square of the coefficient of variation) from a cluster sample may be 
approximated by: 



V2 _ 
X — 



where: 



1 1 +. p (n- 1) 

mn 



2 _ \^ + (1-P) 



V? 
e 

P 
m 

>o 
n 



relvariance, between students, of the amount of error; 
proportion of cases in error; 
number of clusters; 

intra-cluster correlation coefficient; and 
average cluster size. 



Estimates of the coefficient of variation for several values of P are presented in Table 
2-1. The following parameter values were assumed. 



v? 



mn 



.5 (variance estimates from Stage One suggest that the intra-class 
correlation between students in a school is/quite high) 

. 1 (derived from Stage One variance estimates) 

3600 
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TABLE.2-1 

./ 

. EXPECTED COEFFICIENT OF VARIATION FOR AVERAGE. ABSOLUTE 
^ DOLLAR VALUE OF ERROR, ASSUMING DIFFERENT VALUES OF P 

'• ■ ' . . ■ , ^ 

■'<.'■■' ' 

Proportion Of Students W ho Had Some Error 

* .1 

Coefficient of 

variation .052 .065 .085 .127 
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2.2.2 Design Objectives 



In terms of survey implementation, the proposed sample design had|t|:ie following ^ 
objectives: ' 



• Limit the amount of field travel 

• Control the number of students and seps^-ate institutions selected 



• Insure the representation of a variety of institutions ^ 

The procedures described below yielded a sample meeting these objectives. 

2.3 SAMPLING FRAME: INSTITUTIONS USING REGULAR DISBURSEMENT 
SYSTEM 

The sampling frame for the RDS* institution sample was constructed from the 
Pell Grant Program Institution Universe File and the October 1981 Progress Report 
Error/Unreasonable File. The following steps were taken in the construction of the 
frame. 

2.3.1 Selection of a Single Record Per Institution 

A single record per institution was extracted from the Progress Report File, 
which contained progress reports for October and ad Hoc or update reports. The initial 
October report was used unless an udpate was available. Branches of institutions 
reporting through a central office had no Progress Report record because the central^ 



2-6 



ERIC IS 



office accounted for their Pell Grant recipients. Details on the treatment of multi- 
branch campuses where the central office reports for all branches are- given in a later 
Section. ^ - . i 



2.3.2 Selection of Eligible Institutions 



After a single Progress Report record was extracted for each institution, these 
' records were merged with the Universe File, by institution ID^ and only in-scope 
institutions were retained in the frame. . For the' purpose of this survey, institutions 
are considered in-scope if they met the following conditions: 

• They were in the coterminous United States (excludes Puerto Rico, Alaska, 
Hawaii and the Virgin Islands). ' . \ 

i . 

• They had RDS participation codes— position 2^0 on Universe File— of either 
1 (participating, independent campus), 2 (central office for participating 
branch campus sytem), or 3 (branch campus participating through a central 
office). 

• They had an eligibility cTode— position 239 on Universe File— of 1 (eligible 
for Pell Grant). 

• They had an institutional status code— position ^9^ on Universe File— of 1 
(active institution which may or may not be funded). 

2.3.3. Imputation of Recipients (Measure of Size) 

Eligible institutions which were on the Universe File but not on the Progress 
Report File— other than those reporting through a central office— were flagged for 
imputation of the number of recipients. Two hundred and forty-^iwo Institutions were 
in this group. Based on ^,676 institutions that had data on the number of recipients, 
undergraduate enrollment, institution type and institution control. We ^estimated a . 
regression function to predict recipients as a function of the other' three Variables. In 
the estimation, we generated* dummy variables for institution type apd control 
categories and used the most general model, which included two-wjay and three-way 
interactions. The R obtained was .72 (proportion of the variance in the dependent 
variable explained by the regression); a very good fit for cross-section data. Sixty- 
three institutions had no reported recipients and no undergraduate enrollment figure 
and thus imputation with the regression function was not possible. These institutions 
were listed for inspection, and since they were fairly small schools (such as 

y / 
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"Pedigree School of Dog Grcpoming"), they were assigned a value of 8 recipients to be 
used as measure of size, . • 

2.3 A Treatment of Ca|npuses where the Central Office Reports for all Branches 

Institutions where the central office reported for all the branches (Participation 
Code = 2) wefe handled as follows: 

• For campuses broken down by branches in the "Education Directory - 
Colleges and Universities 1981-82", we allocated the recipients reported by 
the central office among the main campus and branches according to their 
respe^jCtive enrollments; that is: 

ReciprientSg = (Enrollmentg/To^l Enrollment) x Total Recipients 

• For other institutions— if the central office was selected, we called the 
school and d^|ermined if records of Pell Grant recipients were kept at the 
central office or at the individual branches, Jf records were kept at the 
central office, we sampled students .from the central office filesf and 
individual branches were not distinguished, 

• / ^or institutions that kept records at the individual branches, we obtained 
the breakdown of number of participants by branch and selected one branch 
within the institution, probability proportional to size (PPS), using the 
recipients at the branch as the measure of size, ^ 

2.3.5 Determination of Certainty Institutions 

ft 

To obtain a final sample of about 300 cooperating schools— allowing for 2 percent 
out^of-scope institutions and 95 percent institution cooperation— requires an initial 
sample of 322 institutions. Institutions with reported recipients greater than the 
overall institution selection interval (total measure of size/322) were drawn into the 
sample with certainty. 

The certainty cutoff was set at 3,7^0 recipients or approximately 70 percent of 
the school selection interval. That is. 



Total Recipients x .70 = 1,7»6,131 x .70 = 37*0 
School Sample 322 
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certainty cutoff yielded 3^ certaihty institutions, ^ of which were central offices. 
^2.3.6 Selectiori of Noncertainty Institutions ^ ' 

The sample design for the noncertainty portion of the sampfe was basically a 
double sampling procedure with PPS selection of flusters and implicitly stratified PPS 
selection of all schools included in the sample clusters. The sample design calJed for: 

• Ordering the file by geographic code and forming clusters of consecfutive 
scJjools, from the ordered frame of about 8 schools each ; 

• Sampling dlusters with probability ^oportion^to total recipients in the 
clusters * , 

• Assigning an adjusted measure of size (AMOS) to schools within sampled 
clusters where: ^ 

AMOS.. = IMOSj/CMOSj \ 
i = school; 

j = cluster; . 
. IMOS = the institution measure of size; and 
CMOS = the cluster measure of size ' 

• Ordering the schools sample clusters, ignoring the cluster identifier, by the 
following type and control strata: 

School control 

Public ' 

Private 

Proprietary 

School type • ' 

Offering 1 to 2 year programs 

Offering 2 to 3 year programs , 

Offering ^ year (or more) programs 

• Sampling schools systematically from the ordered file with probability 
proportional to the adjusted measure of size (AMOS) 



To limit the amount of field travel, we clustered institutions by geographically 
contiguous three-digit zip codes. (A previous step verified zip code and flagged those 



requiring correction.) We defined a. cluster as a set of 8 consecutive schools with total 
minimum measure of size to insure an average of 12 recipients per institution. The 
minimum measure of size was achieved by all clusters. The clustering process resulted 
in 632 clusters of 8 schools each, with an average of 2,^70 recipients per cluster. 

Sample of Clusters, but of tlTfe\j532 clusters in the frame, we sampled 72 
clusters, PPS, with total number of recipients in the cluster as the measure of size 
(CMOS). The 72/sampled clusters contained 576 schools. None of the clusters sizes 
exceeded the sampling interval of Zl,7^13 which would have required their selection 
with certainty at this stage. 

Sample of Institutions. To insure adequate representation of different institu- 
tions, they were stratified by control (public, private and proprietary) and type (less 
than 2 years, 2 to* 3 years, and * years or more). The measure of size for PPS selection 
was the ratio of the institution's MOS to the measure of size of the clusteV from which 
the institution came (CMOS). - That is, . ; 

^ . AMOS.j = IMOS.j/CMOSj • 

as defined above. The 576 institutions in the 72 sample clusters were sorted by the 
type and control strata defined above and a systematic sample of 288 institutions was 
selected with probability proportional to the adjusted measure of size (AMOS). A 
considerable number of institutions (171 or 59.3%) had'a measure of size larger than 
the sampling interval and were conditional certainty institutions; these were removed 
from the frame before the 117 noncertainty institutions were drawn. 

The resulting distribution of the sample among type-control strata was propor- 
tional to the distribution of the universe measure of size among the strata. This is 
illustrated in Table 2-2. 

^A SELECTION OF STUDENTS FROM SAMPLE RDS INSTITUTIONS 

To obtain a sample of about 3,600 Pell Grant recipients— 300 institutions with arT^ 
average of 12 students per school— required an initial sample of about 'tjO'tO recipients, 
assuming a 90 percent combined institution/recipient response rate. The overall 
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TABLE2-2 ' . 

. . • 

DISTRIBUTION OF RECIPIENTS II^ INSTITUTION UNIVERSE 
/ COMPARED TO SAMPLED INSTITUTIONS 



J. 



Universe MOS 
Sample cpunt 



2 to 3 years 



Institutidh 
Type _ 



Less than 2 years 



Universe MOS 
Sample count . 

^ years or more 



Universe MOS 
Sample count 



Total 



Universe MOS 
Sample count 



Public 



Value 



10,3^3 
2 



'f51,H3 
SS 



759,500 
111 



1,221,286 
201 



percent 



.6 



25.9 
27.3 



'^3.5 
3it.5 



7o.O 
62. if 



Private 

V 



Value 



3,126 



3'f,663 
10 



c 



3^*2,776 
7it 



380,565 
8f 



Percent 



18 



1.9 
3.1 



19.6 
23.0 



21.7 
26.1 



ftroprietary 



Value 



92,701^ 
22 



/f6,'fl6 
If 



5,160 
1 



l'f'f,280 
37 



Percent 



5.3 
6.8 



2.7 
f.3 



Total 



Value 



.3 

.3 



106,173 
2tt 



5^2,522 
112 



1,107,^*36 
186 



8.3 



1,746,131 
322 



Percent 



6.1 
7.5 



30.5^ 
3f.7 



63.* 
57.8 



100.0 
100.0 
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recipient sampling fraction, required to achieve an equal probabflity- sample of the 
de^sired size is given by: 

n 



f = "I = = 1 



™OS 17*6131 *32.21 

^ « . *. . 

where 

^ f = overall sampling fraction 

TMOS = total number of recipients (total measure of size) 
computed from the October 198^1 FjlFogress Report File 

"^I = the required initial sample of recipients 

Consequently, the weight associated with each sample recipient— the reciprocal of the 
overall sampling rate— was ^32.21. 

Within any institution, the probability of selecting a given student was equal to 

the overall sampling fraction. Thus, the products of the probabilities of selecting a 

.• • ft 

particular cluster, institution within the cluster, and recipient within the institution 
were equal to 1/^32.21. The sample, then, was effectively self-weighting, with no 
adjustments necessary for institutional characteristics. 

2AA Sampling Frame 

The sampling frame was the list of Pell Grant recipients, at each of the sample 
institutions, fromjvhich the recipient sample was drawn. At the time We contacted 
the school to set up an appointment for the data collector's visit, we requested that 
the school prepare a list of current and pending recipients to be used as a sampling 
frame. 

Recipients were selected from the sampling frame systematically with equal 
probability. For this purpose a sampling worksheet was produced for each institution. 
The sampling worksheet included all the information necessary for drawing the sample, 
such as sample interval and pregenerated selection numbers. * 

• ■■■ 

After the data collectors obtained a list that included all Pell Grant recipients 
eligible for the study, they numbered the students on the list sequentially. Then, they 



selected those students with sequence numbers corresponding to the selection numbers 
given on the sampling worksheet. 



2.^.2 Weightea Estimates , 

7fTe* sa.mple of Pell Grant Tecipients frop RDS schools is an equal probability 
•self-weighting sample; before adjustments for nonresponse all student^ have the same 
weight." The procedures used for determining nonresponse weights have been described 
in Volume 1. The sample selection procedure was designed to ^be used %i providing 
estimates of student characteristics. ' Thus/ estimates fd'r ; institutions are .not 
recommended. 
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INSTITUTIONAL VISITS 

.> ■ " . ■ ■ - a - . ■ ^ , * . ' 

• Institutional visits w^re conducte^d in both th^; fall of 1982 ^nd the spring of 
1983. The fail visits had two purposes: to draw the sample of students who -wduld, as 
'well §s their parents, be interviewed in the spring;.and to compile minimal information 
on newly-instituted institutional validation activities. The results of the latter 
inveistigation were delivered to the Department of Education in December, 1982. The 
methods for training, scheduling, an(d' managing the fall data collectors were, generally 
similar to those reported here for the spring visits. 

Two previous reports on data collection have been submitted to the Department 
of Education, one on training and qualifications of dSta collectors and one on the data 
collection itself.^ Those reports contain copies of forms used, interview ^tipts, and 
resumes of' the data- collectors. This chapter is a compilation of all relevant 
procedural "^nJf methodological septioias fromi those "reports. ^ 

• 3,1 DATA COLLECTORS * ' - * : * 



^•ThQ^data collectors were the key to a successful spring data collection. ^Qiven 
the- Qompilexity*- of ' the Pell Grant Program/ .^nti the Varifety^ . Qi documents and 

■ < ■ • ■ , . ' . ' . ■ • ' ■ • i ' . ' . J ■ 

/institutional recordkeeping syjstems likely to be encountered, financial aid experience 
was essentiali v^Since the data collectors would be/working alone ail over the country, 
: (detailed supervision of them would: be.lrnpossiblie; therefore, "they had to' be reliable 
and mature enough to be able to^ake thefr'own d^^cisioris in rnaoy situations.: 



♦Advanced Technology, Inc*, "Report on Training and Institutional Data Collectors for 
Stage Three of tlie Pell Grant Quality Control Study," * April, 1983, and "Report on 
Spring, 1 983^^ Data Collection for Stage Three of the Piell Grant Quality Control 
Study," June, 1983. , . 



3.1.1 Recruitment 

We began recruiting data collecfctors with the people who had performed 
successfully in our two previous data collections in the spring of 1981 and the fall of 
1982. None of the 1981 data collectors was available for data collection in 1983.y 
However, three of the fall, 1982, field staff were available and were rehired. ' ^ 

; We advertised our field staff needs in four newspapers and periodicals: the 
Newsletter of the National Association of Student Financial Aid Administrators 
(NASFAA); the Chronicle of Higher Education ; The New York Times ; and The 
Washington Post . The NASFAA Newsletter provided the greatest number of applicants 
with the type of experience wq were seeking (at lea^t-^. The two newspapers 
advertisements generated few or no useful responses.' When one of the people who had 
been offered a poskipn declined at the last^r^ment, a well-qualified replacement was * 
hired through the personal contacts of Deputy Project Director Richard Tombaugh, 
former e)cecutive secretary of NASFAA and former director of the Student Financial 
Assistance Training Program. 

3. 1.2 Qualifications ^ 

The spring,^ 1 983 data collectors were the best qualified of the three groups used 
so far. Nine of the 15 people^initially sent into the field had been financial aid 
^irectors, 2 at large state institi^tions; 3 others had over 10 years experience each in 
financial aid administration. The two data collector%^without financial aid experience 
had performed successfully as data collectors in the fall of 1982. The 15 original field 
staff included ^ with bachelor's degrees, 10 with master's or first professfional degrees, 
and 1 doctorate; 1 of those with a master's was an active doctoral candidate. The 
degrees in most cases were in educacTioA. 

Therefore, the data collectors were well-equipped by training and experience to 
deal with the variety of dQ$:uments and record systems they would encounter in the 
field, to explain interview questions to financial ;lftd directors, and to understand 
answer^ouched in tethnical finarii^ial aid terminology. They also^^uld find relevant 
d4ta in studeht files quickly and accurately. Tfte training thferi'^ concentrated on 
ensuring consistent 5 use of the data collection instruments, with little need for 
^strilction on the basics of the Pell Grant Rrogram. 



1 



2 s 



3.2 SCHEDULING ^ 

The principal objective of the scheduling was to minimize travel costs while 
visiting all the sample institutions within the six-week data collection period with only 
15 interviewers. We drew up an ideal-^ster schedule to achieve this goal and then 
adjusted it as necessary when we could not visit on our preferred, dates. 

. 

3.2.1 Master Schedule 

The first step in drawing up an ideal master schedule was to estimate how much 
time was ,needed for each institutional visit. Since the interview had a fixed cost f^r 
each visit, the time allowed depended on the number of students sampled. We 
estimated that each interview would t^ke an hour and that each student abstract 
would require about 20^minutes initially, but slightly less for larger numbers.* 
Therefore, we estimated that each data collect^' could handle 1^ students during the 
first and second days at a school, 16imj^e third day, and 20 on the fourth day. We broke 
these estimates down by half days as follows: 

/ 

Number of Sampled Students , Number of Days 

Yi ^ 
1 . 

^- . • 

2 
3 

Once we had estimated the timfe required for efich institutional visit, we grouped 
institutions into clusters that would each require one data collector one wefek to visit. 

* . ■ jQ^'-A't '^^.- 

We had to allow time to travel between cities in makmg these clusters. In s6me cases, 

a data collector coijild stay in a city for a whole week, even being scheduled to do two 

half-day schools in one day. Although the institutional sample had been clustered for 

yftfficipncy in ..both student and parent interviewing and institutional 

4;io, \ ■ 
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visits, certainty schools— those whose large number of Pell recipients made their 
selection for the sample 'certain— could be anywhere and ^some "clusters" were 
scattered geographically. Therefore, some of the institutions we had to schedule were 
in fairly inaccessible or remote places, like Susanville, California; Trinidad, Colorado; 
and Spearf ish. South Dakota; o^ers were, located hundreds of miles from the next 
nearest institution's location, as was Tuscon, Arizona. We usually allowed a full day to 
get from or to one of these places. We assumed that a weekend was sufficient to get^ 
from . any one-week cluster to another. Occasionally, the schools were 'Arranged 
geographically in such a way that we could not avoid giving an interviewer an open 
date even though a whole day was not ne*eded for travel. 

After clustering the institutions into one-week work loads, we grouped the 
clusters into 15 regions of 6 clusters each. Each region represented pr^ data 
collector's assignment. These regions are shown in Figure 3-1. A major consideration 
in this grouping was the creation of loops which would enable a data collector to use 
the same rental car for six weeks and return it to the place of origin, saving drop 
charges, or faking advantage of round-trip air fares. 

Once institutions had been grouped into weeks and weeks into regions, we began 
assigning preferred dates to the institutions. We tried to designate two alternate 
0 dates for each school, but this was^not always practical. Where we had several schools 
to visit in a single city or within a few miles of each other, we could rotate the 
preferred dates throu^ several days or ^ week easily. In other areas, a week's work 
could be treated as a loop itself, with a preferred schedule specifying travel in one 
dii-ection and an alternate schedule involving travel in the reverse direction. The final 
master schedule consisted of a list of all 317 participating institutions with one 
preferred and one or two alternate dates for each. . 

3.2.2 Schedule Confirmation • 

We used Westat's telephone facilities and staff to call all the institutions to 
^'secure '^agreement on the date of our visit. Westat's telephone center includes 
sophisticated switching equipment which selects the cheapest telecommunications 
system (MCI, SPRINT, AT&T, etc.) for each call, sound-proofed carrels for 
interviewers, and monitoring equipment. Westat has available a corps of telephone 
interviewers who are experienced in following a written script, recording answers, and 
tracking individual questionnaires to completion. 

3-* 
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To confirm each scheduled .visit and collect some other information^or our data 
collectors, we developed a script for the Westat telephone interviewers (Figure 3-2). 
If an institution would ,not accept a visit on the preferred date or any alternate date, 
the interviewer asked what dates would tfe acceptable. The script also gave us an 
opportunity to cortf4rm that the data we required would be available, to reiterate our 
requirements, and to give the financial aicl director at each school advance notice of 
some questions for which he might have to compile statistics. 

Westat telephone staff immediately reported to our data collection manager 
(who had a temporary office at Westat's telephone center during the scheduling period) ' 
every case in which a school would not agree to^ a visit on the preferred date. These 
situations required moving other schools to alternate dates or wholesale rescheduling. 
We also had a script for rescheduling calls. Whenever p^ossible, rescheduling was 
limited to schools which had not yet been reached, to minimize the number of changes 
and the number of times individual directors were called back. 

Westat trained the scheduling interviewers. An Advanced Technology 
professional staff member was assigned to monitor the scheduling calls on a rotating 
basis to cover all the interviewers; when he had time, our data collection manager also 
monitored the scheduling calls. The monitors brought rninbr problems to the attention 
of the interviewers immediately; they encountered no major problems. 

A few schools which were difficult to reach w6re called directly by Advanced 
Technology staff after the initial scheduling period had formally closed. After all 
schools had agreed to visit dates,. we compiled an individual schedule for each data 
collector listing the dates, places, and 'times of the visits, the names of the financial 
aid directors, and the numbers of students in the main and control group samples.' . 

3.3 TRAINING 

.Since all the field staff were experienced financial aid administrators or had 
been data collectors for the fall, 1982, validation evaluation, little training was needed 
on the Pell Grant Program. Therefore, it was possible to devote an entire week to 
interviewing techniques and to filling out correctly*a Student Record Abstract (SRA) 
for each student in the main sample. Figure 3-3 is the training schedule. 

r * 
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INSTITUTION CODE 



INSTITUTION NAME 




4" 




0 


r 




City 




State 




TELEPHONE NO. ( ) 









PELL GRANT SCHEDULE CONFIRMATION 
SPRING, 1983 INSTITUTIONAL VISltS 



1. Good (morning/ af ternoon), (Mrs./Mr.) . I am 



, from Westat, Inc., in Rockville, Maryland. Last fall a 

representative of Advanced Technology, Inc., visited ydur institution to collect 
information on a sample of your Pell Grant recipients as part of the Pell Grant 
Quality Control Study. A few weeks ago Dr., Ernst Becker, head of the Division 
of Quality Assurance in the Office of Student Financial Aid, sent you a letter 
describing the spring^data collection and ^e activities Advanced Technology ^ill 
be carrying out at the sample institutions. Have you received that letter and had 
a chance to read it? 

YES (GO TO 3)- 

NO, DONT KNOW, CANT RECALL (CONTINUE BELOW) 

2. During April and M'ay representatives of Advanced Te^chnology, Inc., the prime 
contractor for the study, will be visiting each of the sample institutions to 
interview their financial aid directors in more detail on validation and ways to 
reduce payment error in student financial aid programs. They will again be 
abstracting information from the files of the students selected last fall. Those 
students and their parents are also being interviewed individually; many have 
already been contacted by field staff. . 

* 

3. For planning purposes, we have established a tentative schedule to visit all the 
institutipns this spring and hope that most institutions will cooperate so we can 
meet the Department of Education's deadline for our report. I have a checklist 
of items to ask you regarding our visit to your institution. 

FIGURE 3-2 
SITE VISIT SCHEDULING SCRIPT 
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3a. Our interviewer is presently scheduled to visit your institution on 

at . Is that date and time acceptable to you? 

YES (GO TO 3c) 

NO (IF DATE unacceptable, go to 3b; IF TIME ONLY 

' UNACCEPTABLE, CONTINUE BELOW) 

What time can our visitor arrive to get in a full day's work? 
(RECORD TIME> (GO TO 3c) 

3b. Because our interviewers have so much travelling to do, it is important 
that we be able to visit all the institutions in one city or area on a single 
visit. Pending confirmation from the other institutions in your area, 

^_ or . would be good alternate 

dates for us. Would they be acceptable to you? (IF "YES," GO TO 3c;..IF 
"NO,^ CONTINUE BELOW.) 

Although we would prefer to interview the financia/ aid director at each 
institution at the beginning of our visit, we could talk to an associate or 
assistant director who can answer authoritatively about your institution's 
policies and practices regarding Pell Grant validation and administration 
and packaging of student aid; or we cpuld begin the visit with the student 
record reviews and talk to you later in the day. Could we use either of 
these alternatives to keep to one of the dates I have suggested? 
YES (RECORD WHICH DATE AND TIME) 

NO (CONTINUE BELPW) 



r 



Keeping in mind that we have to keep a very tight schedule, what would be 
the closest acceptable date to the date I originally suggested? I originally 
suggested (RECORD ALTERNATE DATE) 



FIGURE 3-2 (cont'd) 
SITE VISIT SCHEDULING SCRIPT 

er|c ^-^ 



3c. Are there any locaA holidays or school vacation periods that might ^ 
interfere with the date we have agreed to? 

YES (CONTINUE BELOW) ^ 

NO (GO TOW) * 



Is that a local holiday which might affect other institutions in your area, 
or is it specific to your institution? 

HOLIDAY, MIGHT AFFECT OTHER INSTITtJTlONS 

SPECIFIC TO OWN INSTITUTION 

Again, keeping in mind our need to visit several institutions when we are in you 
area what would be the best time for our visit that would be close to the date I 

originally suggested? That date was . 

(RECORD DATE) ' 

To determine enrollment status and the amounts and dates of Pell Grant 
disbursements, our interviewers often have to visit the registrar and bursar, as 
well^as the student aid offic^ Are both the re^istfar and bursar located in the 
same building as your office? 

, YES(GOTOifb) 

NO (CONTINUE BELOW) 

) 

te. Where is the registrar's office? 

(RECORD ANSWER) ■ . 



Where is the bursar's office? 
(RECORD ANSWER) 



We would appreciate your notifying the bursar and registrar of the visit, 
just in case our researcher noeds to talk to them or check their records. 



Are all the records for your institution's students on your campus, o are any of 
them .kept elsewhere, at a central office or branch campus? ' 

. ALL RECORDS KEPT HERE (GO TO QUESTION 6) 

SOME OR ALL RECORDS KEPT ELSEWHERE (CONTINUE WITH 

, f 5a BELOW) 



FIGURE 3-2 (cont'd) 
. SITE VISIT SCHEDULING SCRIPT 
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5a. 



What records are kept elsewhere? 
(RECORD ANSWER) " 



Sbi Where are those records kept? (RECORD NAME AND ADDRESS OF 
OFFICE, INSTITUTION, CONSULTANT, ETC.) 



5c. Whom should we contact to get access to those records? 

NAME 

TELEPHONE NUMBER ' 

5d. Are the records there complete, or wpuld we have to visit your office or. 
campus also? 

• YES, RECORDS THERE ARE COMPLETE 

WOULD HAVE TO VISIT THIS OFFICE ALSO 



Will enrollment and Pell disbursment records be open and available to our visitor 

on (DATE OF SCHEDULED VISIT) ■ ■ ■ ■ ■ ■ ; ? ' 

YES(G0T0 7a) 

NO, PELL RECORDS NOT OPEN ON THAT DATE (GO TO 6a) 

NO, CANNOT PERMIT OUTSIDE INSPECTION (GO TO 6b) 

6a. Again, keeping in mind oucneed t6 visit several institutions when we are in 
your part of the country, what would be the date(s) closest to the date I 
originally suggested on which ^ we could visit. • That date was 



(RECORD DATE) 



FIGURE 3-2 (cont'd) 
SITE VISIT SCHEDULING SCRIPT 
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6b. As the earlier correspondence explained, are under contract to the U.S. 
Department of Education to complete this study, and, under existing 
regulations, information about a student's financiad aid may be disclosed to 
authorized Education Department representatives without the consent of 
parents or students. We did examine Pell Grant records last fall for the 
students we selected then. Is there an official reason or new institutional 
policy which would require you to close the records? 

• YES (RECORD REASON) ^ 



(CLOSE INTERVIEW AND REFER ' INSTITUTION TO PROJECT 
DIRECTOR.) ' 

7a. Can you suggest where our researcher should park his or her- car and note 
any special procedures to be aware of, such as checking itt' with ca^npus 
security? 



7b. Is there any particular nearby hotel or motel which you recommend to \ 
. people from out of town who are visiting youf school? (DO NOT PROBE 
FOR AN3WER.TO THIS QUESTION) ' " ' ' / ' - • --^ " ' 



FIGURE 3-2 (cont'd) 
SITE VISIT SCHEDULING SCRIPT 
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8. Before ending the call, let me just give you some advance notice concerning a 
couple of items our interviewer will be asking about which you might have to 
look up. We will need to know how many students were selected by^ you and by 
the U.S. Department of Education for Pell Grant validation, hoW many you asked 
for citizenship or residency documentation, and hQw many corrections were 
made on the basis of Veterans Administration - ihformation. We will also be 
asking about the effects of validation in t6rms of the number .of students whose 

; payments were delayed and how long the average delay was. ^ 
(IF ^RESPONDENT ASKS FOR DETAILS OR DEFINITIONS, READ THE 
FOLLOWING:) 

Our researcher, who will be an experienced financial aid administrator, will have 
the details. We just want to let you know in a general way about a couple of 
questions that might Tequire you to look up some numbers in order t<i answer 
correctly. ' ^ 

9. Thank you for your help. Let me review the date ^e hiye agreed to: Our 
' researcher will arrive on at ' • ■ .■ •'">' . Is this 

your understanding? . 

"^fES 

^ , NP (RESOLVE) 





■ ... . 1., 0.- ' — 










■ * ■ ' ■ ■ ■ ' ■ ■ ■ 













Our researcher will call you from the field a few days before arrival in order to 
confirm the visit. ' 



FIGURE 3-2 (cont'd) 
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TRAINING PROGRAM .: " 
PELL GRANT QUALITY CONTROIJ STUDY ' 
Monday, April it 
AGENDA ■ 



8:15-8:30 Coffee 



\ ... 



8:30 - 9:15 ' . . . General Welcdme, IhtroduGti6ns,;^Va^ Overview; ^pf 

Traihing Week - * Mr. Richard ^Tbtnbaughj . Advanced 
Technology' ' ■ 

9:15 - 9:30 Introduction tor Advanced T^hnblpgy/ -- Dr. Alex 

Ratnbfsky, Advanced Technology ' / 

9:30 - 9:^5 ^ ^^^^ Quality 
; s Control Study - Mr. Ernst BecKer, Delp.artment of 

V Education . CO 

'9:^5 - 10:00 Overview of Stage III Quality Control i^L^tivities - MS. ' 

^ ^ Carol Miller, Advanced Technology, ; and Mr. Bob 

Learmonth, Westat 

10:00 - 10:15 Overview oi Data Collectors' Tasks aijd Responsibilities - 

V Richard Tombaugh . - 

10:15 - 10:3d^-.> ; . ■ Break 

10:50 - 12:15 • Conducting th^ Institutional Interview With the FA A 

Richard Tombaugh \ 



12:15 . 


- 1:00 


1:00 - 


1:15 


1:15 - 


3:00 


3:00 - 


3:30 


3:30 - 


't:30 


. ^:30 - 


5:30 




LuncH 

Review of Training Manual - Ms. Tracy O'Connor, 
Advanced Techridlogy 

Detailed Item^fcy-Item Examination of Institutional \ 
Interview Form 4 Mr. 3ohn {Nl'e1^|y, Advanced Technology 

Break 

Demonstration df *InstiiM^ronalyinterview - Carol Miller 
and Richard Torn^aij 

Review Answers |i^om Demonstration Interview; Group 
Practice: Beginning the Interview, Answering Respondent 
Questions, and Asking the Interview Questiohis - Carol 
Miller and Richard Tombaugh - > ' \ 



1: ^ ^ FIGURE 3-3 



9:30 - 10:30 



10:30 
10:'t5 



10:'t5 
12:15 



12:15 - lA'-^ 
1:00 - 2:00 



J3 




2:00 - 3:00 

:3:06v'^^:15: 
3:15-4:15 
- 5:00 



5:00 - 5:30 



Tuesday^ April 5 



Coffee and 




'■y '. ' V. 



hotds 



Role Playing TO Institutibnal;. interview Trainees in 
Pairs ' S; :>V-.\.:" 

Acting Out of Institutional Jnteryiev/;^^^^^^^ Playing 

Difficult Respondents >yith Tjraihees^^ to* 

Practice Probing and Handling- pifficul^^^ -Carol ^ 

Miller, Richard Tombaugh, 'and D^^^^^ Parker, ■* 
Advanced Technology • • ; 



Break and ID Photos 



Detailed Item-by-Item Examination - pf Stufient JRecord 

Abstract and Specifics of Completing^ 

Neely 

* ■ * ■ ^ 

Lunch 

Distribution of Hypothetical Student Financial Aid Record , 
File; Perusal of Sample Forms Likely tOvbe Found in 
Student Aid Files; Review of Federal Tax Eorms - Richard 
Tombaugh 

Practice Completing a Student Record Abstract Using 
Data from Hypothetical Student Files" ' . 

Break. -'-s'/:^ ;^ ' ' \ ''^ 

■ Practicie Completing a Second Student Record Abstract 

^iWjstracting the Atypical Student Uecord - Richard 
i tombaugh 

:Xonipletion of Business Forms and Associated /|^aperwork 
- Tracy O'Connor an^flanice Grant 



Evening Assignment: Student Record Abstract (Practice 
Three) ' ^ 



FIGURE 3-3 (cont'd) 
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Wednesday, April 6 



■Coffee ■ ' ' s'i'-'- 

•Review of Answers for Tuesday Evening Assignment - 
Richard Tombaugh 

Practice Completing a Fourth Student Record Abstract 
Break 

Discussion of Institution Control Group Activities - Albert 
Parker 

Role Playing of Resolution/Exit Interview with FAA - 
Carol Miller and Richard Tombaugh 

Lunch , , 

Explanation .of Shipping and .Receiving of Data Collection 
Materials, Editing and Post-Interview Procedures - 3ohh 
Neely ' . ' 

Explanation' of Wha| Hap^ens*^ to Completed <jE>ata 
Collection Forms Upon Return to Project Office: Log-in 
of Forms, ihterview Verification, Coding, Keypunching, 
and Data Processing - 3ames Southwick, Charles 
Schueller, euid Dr. Dan Qeller, Advanced Technology 

Explanation of Tr^^l- Arrangements - - Cash Advances, 
Hotel Reservatici^s^ 7^^^^ Tickets, Car Rental 
Procedures, aod' /Eiipense^ Reports; Discussion of Field 
Reporting and'v^Ttici Problems - t^acy 0*Connor and 
Albert Parkew:': -l:^^- 

Break 

Description of Thursday, April 7, Field Practice - Richard 
Tombaugh 

Practice Completing a Fifth Student Record Abstract 



FIGURE 3-3 (cont'd) - > 
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Thursday, April 7 

Review of Pilbt Data Collection for Campus-Based 
Programs - Richatrd Tombaugh (Justice, Alvarez, and 
Finch only) / / 

Field Practice at the Following Area Institutions: 

American University * ; ; , 

Washington, D.C. ^ - 'v 

Computer Learning Center (afternoon only) • 
Springfield, VA 



George Washington University 
Washington, D.C. 

^ Georgetown University 
Washington, D.C. 

Montgomery Community College 
Rockville, MD 

Montgomery Community College 
Takoma Park, MD 

Howard University 
Washington, D.C. 



O 



FIGURE 3-3 (cont'd) ^ 
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Friday, April 8 

8:15 - 8:30 « Coffee 

8:30 - 10:00 Detailed Review of Previous Day's Field Practice - - 

Discussions of Experience, Answers to Trainee Questions, 
Advice on How to Handle Problem Situations - Richard 
Tombaugh 



10:00 - -10:15 SUmmary - Carol Miller 

10:15 - 11:^5 Confirmation Calls td First Week's Institutions 

11:*5 - 12:^^5 Lunch 

12:*5 - 2:00 Issuing of Cash Advances; Trip-^to Bank for Traveler's 

Checks 

2:00 - 3:00 Recap of Training Week; Completion of Paperwork; 

Answering of Questions 




FIGURE 3-3 (cont'd) 
TRAINING SCHEDULE 
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3.3.1 Manuals *^ 

We produced twq^ instruction books for the data collectors: a Data Collector 
Training Manual and Question«by«Question Specifications . We also distributed to the 
data collectors for reference in the field the Pell Grant Validation Handbook and two 
booklets published for OSFA's Student Financial Assistance Training Program: Aid 
Administrator's Guide to IRS Forms and Schedules and Index of Regulations . The 
Training Manual covered general data collection procedures, specific procedures for 
this study, and travel and accounting arrangements, but not how to code specific items 
on the data collection forms; that was the province of the Question-by-Question 
Specifications . 

3.3.2 Interviewer Trsdning 

Since our data collectors had considerable experience in reviewing Student Aid 
Reports (SARs), income tax forms, and other documents in student aid files, but little 
or no experience in conducting structured interviews, we devoted a greater proportion 
of the training to the interview than the amount of time required for it in the field 
would suggest. 

During the first day of the training week, we introduced the data collectors to 
general interviewing procedures with a sound and slide presentation developed by 
Westat for interviewer training, made a detailed, iterh-by-item presentation of the 
institutional interview form, and staged a demonstration interview with senior project 
sta/f playing the parts of interviewer and financial aid director. The second day of 
training included opportunities for each data collector to practice an interview, with 
another data collector playing the role of financial aid director, and a demonstration 
by project staff of how to handle three types of difficult situations: a hostile, 
uncooperative respondent, a voluble respondent who wandered off into long but 
irrelevant answers, and a taciturn respondent who had to be asked to expand or explain 
brief, cryptic answers. We also staged a demonstration of the exit interview in which 
data collectors were to attempt to resolve certain problems which they had uncovered 
when they filled out the Student Record Abstracts (SRAs). This demonstration was 
important because ?here was no specified form or script for this exit interview. 
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3.3.3 Student Record Abstract Training 

Although our field staff had plenty of experience with the student aid forms and 
validation documents from which they were to collect the data on individual students, 
they needed training in filling out and editing our form^^so thatieveryone would collect 
t^e same information in a format that could be coded easily at the project office. 
Therefore, we went over the SRA in detail, one item at a time, defining exactly what 
information we wanted, explaining what documentation we desired, discussing possible 
problems, and answering questions. The most important part of the training was a 
series of exercises in which we gave the data^ collectors student aid files for fictional 
students from which to practice filling out SRAs. We then reviewed the data and how 
they should have been entered on the form. Mr. Richard Tombaugh also conducted a 
special training session on some of the unusual situations which might be encounterieds^ 
in^he field. 

3.3.* Site Visit Practice > \. ' , 

v" • . • ■ . "z t ; ' • v.* ' ^' 

•Qn the fourth day of the training^eek, we rent the data' collectocsJn* groups- b^^^^ 
two or three to seven local institutions wWth were not included :W t|^e sample;;each * 
group was accompanied by a member of the project staff. The purpose of these visits 
was to give the data collectors an opportunity to practice the interview and 
completion of several SRAs under conditions which more closely approximated what 
they would encounter in the field. 

We had arranged beforehand to interview the financial aid. director or Pell Grant 
manager at each practice site. Each of the two or three data collectors did part of 
the interview. We had also asked the financial aid director to pull several files at 
random for each data collector. These were not a scientific sample but did provide 
practice with real files and, at some sites, computerized record systfems. 

On the day after the practice, the last day of the training, we held a discussion 
and evaluation of the practice at which the data collectors shared the lessons they had 
learned and the project staff answered questions. 
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Extensive field supervision of the data collection staff ensured that we became 
aware of problems and resolved them promptly. Field supervision included regularly 
scheduled calls to the field supervisor from the data collectors; unscheduled calls to 
the field supervisor for the resolution of specific problems; field visits to all data, 
collectors by senior project staff; validation calls to a sample of the institutions 
^visited by each data collector; and calls to data collectors for resolution of problems 
revealed by editing. 

3.^.1 Scheduled Calls 

Each data collector had a regular time at which to call the field supervisor each 
week, regardless of time zone - or whatever else the data collector was doing. 
(Exceptions were made only for interviews which interfered with the call because of 
time zone differential.) The scheduled call was an opportunity for the data collector to 
raise any problems which had not required immediate consultation with the project 
staff, including prospective problems uncovered by calls to institutions to confirm 
appointments. It also gave the field supervisor a chance to discuss problems with the 
data collector, especially those which had appeared during editing at the project office 
but were not serious enough to demand an immediate call, such as items omitted on 
the SRA which required a call from the d|^a collector to the institution. The field 
supervisor also informed the data collectors abolit errors in the completion of SRAs 
which did not require calling the institutions for further data, such as inadequate field 
editing. The coders had forms for this purpose which they could place in each data 
collator's file. The field supervisor h^d a checklist of potential problem areas to 
review with the data collectors each week. 

3.^.2 Technical Liaison ^ . 

It 

i^Calls from the data collectors to the field supervisor were more frequent than 
regujarly scheduled calls, especially during the early weeks of the study. These dealt 
with unique or unusual situations at particular institutions which had not been covered 
during training, and concerned the SRA almost exclusively. How to answer a 
particular question often depended on the purpose of the question or how the data, 
would be used in analysis, so the field supervisor checked often with the manager for 
data analysis or other project analysts. 
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3.4.3 Monitoring 

A member of ^the project management or analysis staff visited every one of the 
data collectors during the first eight days in the field to observe how thfey interviewed 
financial aid directors and completed SRAs. The visits were made by the project 
manager, the deputy project manager, the manager for analysis, and a project analyst. 
They monitored the new data collectors (those who had not worked on the project i^ 
the fall of 1982) during the firi^t week, and, the .three returning data collectors on the 
first three days of the second week*. 

During these monito'ring visits, the monitors answered questions which had not 
arisen during training,^^^ some minor errors in completibrt of the SRAs, and 

,rnade other suggestions to the data collectors. They fourid no systematic errors or any 
pr'oblems comparable to those which had led us to'puU a data collector from the field 
during the ,f all. : ' < - -V ^ 

3.4.4 Validation 

Monitoring visit^JvJM' Itob expensive to do more than once as long as no major 
problems were discover^^^ ' As a continuing check on the performance of the data 
collectors and a source of ,;feedback from a different ^perspective, we instituted 
telephone vaHdation. In^^^iecond week of the field period, a professional project 
staff member^dalfea of the financial aid directors who had been visiisd' 

during the ^'vTpt V^ek an4yy|^ a brief questionnaire which included an open- 

ended J^ei^it ||bQut -Collector's performance (Figure 3-4). Each -'Week 

thereaft^P,%e^(b^^ the financial aid directors who had been visited in 

the previ6v>§-*vei 

No pro}3|?ms^:^;^^ this monitoring. The responses to the final 

open-ended q^^'^^ knowledgeability, and coopera- 

tiveness of thi^ldate^?t^ of the financial aid administrators who were 

interviewed raised ^riy t)^^^^^^ data collectors or objected to the way they 

did their jobs. . A ^ J i <^ ' > ' - • 
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INTERVIEWER VALIDATION REPORT 



Interviewer 

Institution Visited 



Financial Aid Administrator, 



Telephone Number 
Date of Visit 



Validation Calls (Enter call-back time in next column) 

Date: 9 

Time: • 



Hello, this is (YOUR NAME) from Advanced Technology, Inc. In order to assure 
the quality of the data collected for the Pell Grant study we are doing for the 
Department of Education, we are calling the participating institutions to evafuate the 
performance of our field personnel. 

On , was scheduled to interview 

Date of visit Name of Interviewer 

you and collect some data from some pf your student-files. 

1. Did (he/she) arrive on time? 

1. Yes ^ ■ ( ' 

2. No - : ' 

■ . ■ ^ 

2. Did (he/she) present (his/her) credentials? 

1. Yes ' . . 

2. No 

3. Did (He/she) conduct the interview with you in a professional manner? 

■ / 

1. Yes 

2. No — --What in particular did you find unprofessional about (his/her) 



conduct? 



FIGURE 3-it 
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Was (he/she) able to answer any questions you or your staff had about the study? 

1. Yes ' 

2. No ^What was the question (he/she) could not answer? 



Was (he/she) careful with your files and cooperative with your staff? 

1. Yes 

2. No — What problems did (he/she) cause? _^ . 



Did (he/she) conduct an exit interview with you or one of your professional staff, 
or offer to? 

1. Yes ^ <^ 

' 2. No 

Do you have any further comments about (his/her) performance, or any 
suggestions to pass on to (him/her)? (RECORD COMMENTS) 



■ f 



FIGURES^ (cont'd).^ 
INTERVIEWER VALIDATION REPORT 
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3.^.5 Problem Resolution - • 

.1 " . 

Whenever possibte, data collection problems revealed by editing of SRAs were 
resolved through calls to the responsible data collectors. Sometimes they remembered 
the circumstances at a particular institution and were able to provide additional 
information immediat||y. In other, cases, they hadrto call the institutions themselves. 
Having the dSta collector, rather than a member of the central project staff, call the 
institution had two adyantag^es. First, the .data collector was already known at the 
institution, knew which specific office or staff member to call, and knew the records 
system^ Second, going back to the data collectors emphasized to them the seriousness 
of getting complete and accurate data and created an incentive for avoiding future 
mistakes. 

the most common problem which required recontacting the school was ambiguity" 
in cost-of-attendance data. Some schools had unusual ways of calculating this figure 
which were not absolutely clear to data collectors during the visit. In a\^ew cases, 
necessary SR A data had not been reported in full. 

■ ■ ^ 

3.4.6 Financial Reporting and Cost Management . 

Advanced Technology instituted a set of strict cost-reimbursement policies and- 
explained them to the data collectors during training. Data collectors were required 
to submit comprehensive weekly expense statements with receipts and daily expense 
logs. . When these reports were late, pay checks were not forwarded to data collectors' 
accounts. In a few instances, the fieldysupervisor asked about unusual expenses during 
weekly calls; he also notified data collectors about non^reimbursable expenses at that 
time! Non-reimbursable expensed were Vare and consisted mostly of personal 
telephone calls which had gone over the daily limit. ? 

We required data collectors to notify us of travel changes beforehand. ^Most 
changes' were to the financial advantage of the project. Because we had jecruited 
data collectors nationally, we had several who were working near their homes and 
found it convenient to go home on weekends, saving us hotel -bills and reimbursement 
for meals. Once they got to the cities they would be visiting, some data collectors- 
found hotels that were both closer to their schools and cheaper than the ones our 
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travel agent hdd been able to reserve at a distance. In general, the field staff were 
quite costTeconscipus and were willing to save the study money when they could. 

3.4.7 Schedule^^ 

Because :the initial scheduling round had required changes in^the^dates for some 
schools^sometimes more than once— we required approval by the field supervisor (who 
had also managed the scheduling) for any field/Changes. Data collectors in general 
were busy enough that they needed all the time we had allocated for each visit, anfJ/^ 
they rarely req[uested permission ;to change their schedules. , 'W^ granted approval for 
minor schedule changes in three in<itances. ' ^ 

3.5 EXPANSION OF^ONTROL GROUP DATA ' 

After the study had gone into the field, ED requested that more data be 
collected on the control group sample than had been planned and than the control 
group data collection form would accominodate. Advanced Technology agreed to 
collect these data by using selected questions from the Student Record Abstract, 
beginning with the institutions scheduled to, be visited on Monday, April IS, the 
beginning of the second week of data collection. 

: We were able to telephone all 15 field staff to tell them about this change in 
procedure. We carefully explained which questions of the SRA should be answered for 
the control group students. We also sent a printed field memorandum (Figure 3-5), but 
this could not reach all the data collectors by AfH-il 18. 

1.4. 

The data collectors were reporting difficulty in completing their assignments 
within the time allowed; the extra time required to fill out a large part of the SRA 
rather than the original control group- form promised to prevent them from 
maintaining their schedules. Therefore, we made arrangements to hire and train 
auxiliary data collectors to make separate visits to ahy institutions with control group 
samples of five or more and complete the cofitrol group data collection there. 
Through the deputy project director's personal contacts, we hired a retired financiar 
aid director and one' whose school was temporarily closed. They came to the project 
office for three days of training which included a field practice and covered the entire 



PELL GRANT QUALITY CONTROL PROJECT . . ' ' ' 

••: SPRING, 1983 DATA COLLECTION ' ■ 

;/ FIELD MEMO #3 

, ' ,The, following items make explicit the new procedure for the I^^ 
GrbuptlCG). ^ . ; ? 

• ^ Draw field samples as originally planned, , ^ 

• Do not fill out the white, *-page, ICG form; instead, used the blank SRA$, 
filling out the following items: 

-> QlrQS (all Of page 2) 

Ql'f' from the ICG (write or tape k) inside of front cover facing page 

■ • :^:;\-2)-'-v^..y - -, ''^ 

■ ^: - -'^ v'Q45-Q/f7 (bottom of page 12) ^ 

. Qifg;,Q56;(s€^CtlorilI^ ; . . . • 

• Copy the student name, student study ID, and student's SSN (Items li 3i' 4) 
frorfa the ICG to the front of the SRA. 

• Write :"ICG" in the top right corner on the front of each SRA you use' in 
plsJbe of an ICG form. . 

• w4 will plan to send an "aide" to any school wher^e you have 5 or rnore ICG 
students already selected. Aides will go, into the; field on May 2. We will 
try to get aides to schools with 5 or more. IC^s .dijSring your visits; schools 

^ yo\visited between April 18 and April 2^ Vill be Visited by aides between 
May>Bnd May 20. . 

• We^will ,fc|an on your completing SRAs^n place of ICG forms at any school 
wher*fe^pu have 4 or i ewer ICGs. • 

• If you have time to do all the ][^Gs at a school with 5 or more, please do so 
and tell Albert Parker about it, . ' 

• Since pD may change its mind again, do not dispose of ICGs until you have 
completed a school; we may switch back to them. Do not send unused ICGs 
to the project office. 
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CONTROL GROUP DATA COLLECTION 
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SRA, equipping, them to assi^ the primary data collectors with the main sample when 

necessjary,;.;.';;;' ■ , 

We had a tentative schedule for these auxiliary' data collectors whifch consisted ,^ 
mostly of schools which. had not yet' been visited by the primary data collectors. It 
eventually developed that the primary data collectors were able to complete all of 
their control group forms at many of the schools where we had expected them to need 
-heli>*; Evidently,^ they became experienced enough with the SRA to have sufficient 
time to undertake the added burden of the, expanded control group data. In order 'to , 
. reduce trayel time and costs and limit data collector travel fatigue, we tried to 
schedule the auxiliary visits to the remaining large Schools on a regional basis. This; 
required in two instances that the auxiliary data collectpr yisit^ the institution before 
/ . the primary data collector did the. interview. At six institutio^, the auxiliary data 
. ^ collector's .^sit was concurrent TXrith the primary data collector's. At another four ' . 
institutions, the auxiliary data tallector completed the control group' forms after the 
, \ primairy data coUectbr v' .. ; ^ 

3.6 ' FIELD PERSONNEL PROBLEMS 



' During the data collection period, several problems arose which required . 
personnel reassignment and other, management During the recruitmejnt.of 

^ v data collectors, we had identified one fully-qualified candidate who was und^ble to 
spend a full six weeks in tlie field but who/ was available, f or shbrt-term-emergency:^. 
situations; we hiade extensive use of her time. 

One of the candidates, whom we hajj offered, a position declined it on Vhe 
. Thursday before the training week. Although we w^re able to recruit a replacement, 
he could not go into the field until the Friday of the second week .of data, collection 
due to other commitments. He participated in the training and the substitute covered 
his schools for the first nine working days. 

-'^^ ■ ;■. ■ ' ~ ^ "• ' ' ' "v:. 

■ In the middle of the third week,. a data collector notified us that she would hav^ 
to resign because of the critical illness of^a family member. Our substitute took oVer 
h^r region for the fpurth-.and 
. • reluctant to stay in the field for three full- weeks. ^ ^ The decreased need lbr>t|?e 
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sauxiliary data collectors was Evident by then; we trained one of tfiem in interviewing 
ti'as well as SR A completion and scheduled him to fill ift -fof the fifth^week of the* field 
period.-^ • 

■ ' ' ' ■ . 

One of 'our data»x;611e,ctors became*ill herself. She was able to get; home at the 
^ end of the third week, and was a'dvised by her physician not to work for a few days at 
^ast. We sent a proji^ct analyst to do the data cdjfecti^n- at her next institution. 

For:tunately, she recovered'^and was able to resume her s|h^dule. after missing only 

that one institution. V 

€fc r • ; • - 

3.7 DEBRIEFING 



We brought all but one of the data collectors back. to 'the project office on the 
Saturday after the last week of the field' period for a ^^ebriefing (the absent data 
collector had a family emergency but* submitted an extensive written report). Dr. 
.David Iwamoto and^Ms. Jean Saunders frdrn'ttie Department of Education observed the 
morning sessions, which were concerned with substantive Pell Grant issues. Seven 
members of the Advanced Technology professional project staff also attended. ^ 



About a week before' the debri^iing each) data collector had' received a fprmal 
debriefing agenda (Figure 3-6) which listed the^topics for, discussion. During regular 
telephone supervfeiDn' calls, \ve a5k«a them all to make informal, written notes of what 
they thought >they' should talk about at'Tthe debriefing. These notes would focus their 
attention and stimulate them to organize their thoughts, thus enabling them to usevthe*, 
debr^iing time more efficiently. The notes' would also reduce the possibility that 
important items wodljd^be bv^rlooked or omitted. We collected notes from nine of the 
fifteen participants. 

The debriefing agenda was : designed to permit' free discussion by the data 
collectors while ensuririg that all important topics were covered. Each session was 
moderated by the* project staff member most clpsely concerned with the topic. thrcJltigh 
experience or project responsibility. Session moderafors intervened when necessary to 
give every data collector a chance to participate and direct the di'scussion to agenda 
topics that v/^uld not otherwise have arisen. ^ 
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. PELL GRANT QUALITY CONTROL PROJECT 
SPRING, 1983 DATA COLLECTION, 

DEBRIEFING* AGENDA ■ . . ' 
Saturday, May 21, 1983 

8:30 - 8;35 Introd^iis^iQiLand purpose of debriefing - Carol Miller 

8:35 - 9:15 *Institutional Validation - Dan Geller ^ 

* • Extent of institution^ validation * 

. - Which schools do and \j(hich don't ^ "# 
What types of students 
a • ^'Exemplary procedures 
f Non-exempjar^ procedures (what nqt to' do) 

V ' . - " , - 

9:15 - 9:'f5 ♦Institutional Error - Dan Geller 

• Award calculation error, including payment schedule look-up 

• Monitoring satisfactory academic progress and enrollment 
status ' ' ...^ 

3i(^5 - 10:00 . ♦Financial Aid Records - Dan Geller * 

Completeness of records at f inancial aid office 
Paper records 

- Access tp computer files maintained by other offices 

• Cornpleteness of validation documentation 

Tax returns 

Social Security and VA benefits.. 

- v' Other documentation 

10:00- 10:15 BREAK 
10:15- 10:^^5 ' ♦Disbursement$-r John Neely 

• Frequency and method of disbursements [J 

• .Recovery of overpayments 

• Accommodation of validation delays 
^^i'-rt Extent of payment delays 

Methods of accommodatiorf ' 

10:^*5- 11:15 ♦Financial Aid Packaging - John Neely 

^ General reactions by FAAs to packaging questions ^ 

i Variations in use of hon-Pell aid * ' 

• Adjustments fob Pell overa^/ards and undera^ards 
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12:W- .-1:00 
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1:15 - 1 
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♦Campus-Based Programs^ Richarc^Tombaugh 

• * Differences between Pell and Campus-Based 
^ «ft Additional observations about Campus-Based 

LUNCH 

Data Collection Instruments - John Neely 

j0 , Pr9bletn' items 

• Format improvements 
• • ICG Forms ,; . 

, Training E>ro|ram - Richard Tombaugh 

» 

• ^Effectiveness of training materials: ^ 
Manuals 

- Exercises 

Practice Visit 
Effectiveness of training on: 

Conducting the interviews 
Dealing with different kinds of records 
Clock-hour schools' 
Difficult respondents and situatipns 
' - Answering questions about the study 

Analysis of Data - Dan Geller 

• Summary of findings to date 

• Quality of data coded so far 

Logl^ics - Albert Parker 

k> J Itineraries (hotel, air, and car arrangements), schedules 



: J • Supervision 
' , ' Field monitoring 

- - Validation feedback ' 
Call-in procedures 
^ • Material suppoct 

Expense advances and paychecks 
Field memos ' 
-^^ - Forms resupply 

1 :^5 - Completion Final Buajiyftss Close-Out - Tracy O'Connor 



• ' Reconciliation of final expense repof^ts 

• Collection of unused tickets and materials 'and 6i car rental 
recards • .jr ^ 

• Distribution of homeward airline tickets 



♦These sessions will be documented- 
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^ T?ie substantive morning sessions were taped. We compiled a transcript of these 
sessions to augment the written notes submitted by the data collectors and to use as 
an additional source in program analysis. A project staff member also took extensive 
written notes during the entire debriefing. The morning debriefing sessions were the 
subject of a report, which included the transcript of all the taped sessions.* 

The afternoon debriefing sessions were devoted to topics relating to the 
administration of the data collection— instrumentation, training, and logistics— and to 
final business arrangements with the data collectors. The afternoon sessions will 
provided a basis for improvement of future data collection efforts. 



^Advanced Technology, Inc.,"^*Report on Debriefing of Spring Data Collectors for 
Stage Three of the Pell Grant Quality Control Study," June, 1983. 
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STUDENT AND PARENT DATA COLLECTION 



Westat collected data during the Spring of 1983 from a nationally representative 
sample of 'f,109 Pell Grant recipients and their parents. The sample of students was 
selected from 317 participating institutions. 

chapter discusses the following aspects of the field data collection effort: 

Organization of the field force 
Supervisory and interviewer training 
Field operations 

The Automated Survey Control System (ASCS) 

*.l ORGANIZATION OF THE FIELD FORCE 

To manage the Pell Grant interviewing staff and to coordinate field operations, 
Westat divided the ^8 states into 7 supervisory regions (see Figure ^-1). Five of the 
seven supervisors had also been supervisors on the previous round of Pell Grant 
(formerly BEOG) data collection in 1981. The other two supervisors had extensive 
interviewing and supervisory experience on other, similar studies. The supervisors 
reported directly to the Westat field director, who in. turn, reported to the Westat 
project director. > 

Regional supervisors and their assistants began recruiting interviewers in 
December, 1982. Primary sources for recruitment included Westat!^:omputerized 
interviewer file, including the file of previous BEOG interviewers; supervisors' local 
contacts; local emjSloyment'kgencies; and, when necessary, rtewspaper ^UVertisements. 
The location of sampled institutions and the number (butVnot necessarfly the location) 
of students 'and parents associated with each institution^ere known at the time of 
recruitment. Westat used this information as a basis ^.Jor identifying desirable 

r, 
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locations for interviews. When, intehv^ began on February 1^, 1983, 1^2 

interviewers had been recruitet;! an^ jnvited tp training. 

tt.2 SUPERVISOR AND INTERVIEWER. tRAINING 

' : ' " ■ : ' ' ' . " 

Westat carried out a training 'p^ its home office for regional supervisors. 

Training sessions for indivldqaii interview were held at four sites throughpqt the 
United States. .'•^ ^'"■•r.-/' "/":* v^'^ 

The five supervisbl->, wh on the previous round of the Pell Grant 

Study were sent the s'tud^yVm the revised Supervisor's Manual . They were 

instructed to review /and; ^tudy and discuss any questions or problems 

with the field direetcir,.': ; , , • . V 

The 'two new supervisors ca Westat's corporate . headquarters for training on 
Febriiary 7-9, 1983^ .',Ti^ey alsa^ sent study materials ti^'T^^iew ^rior to coming for J 
training. . At the training, pto^ reviewed)^het;,histpr^ purposes of the study, ' 

V discuSsed all study mat^^d reispdjWbilities of .^he^upervisor. vftn 

important part of training was instructing the superviWts on hqW^4 ASCS and- 

-giving' them tinrv^ to^^^^^ the system. TJfe finals d^^ with the' 

Westat supervisor 'in/ Balt^^^ This field tripy^as^^ 

^supervisors the bpportUiiity how an experienced supiefyi^br prg^^ office 

for the Pell QrantVstu<Jy;:and to discuss with her thie'iprobiems: the 
; supervisor's; job.:/; / ^ / '■ ■•^^^^ - 

I The goal: of Westat^s^^^^^^^ training sessions was to d($yel^ 

interviewing skills; ol the po^^^ interviewers to enable them to cdnect;^^^ 
in a uniform and ' professional manner. Interviewers wer^ instruq^^^^ 
areas: general interviewing techniques, field procedures specific to this -stM^ 
unde^rstanding of the intent and use of the survey instruments. The greatest . iarripunto^^ 
traihing tijhnie 'wais: devot to questionnaire-related issues, such as^^resent^tibhs- 
question specifications a^ role playing of practice ifiteryiews,/ to which 

.mock res{3ondents used a scripted answer guide. ^ 

. ■ \ ; A total of Hi trainees were trained in one of four sessions, lasting fourv 

• days. During the first week (February 15-18), training was h^ld in Philadelphia and, St.: 



Louis; during the second week (February 22-25), training was held in Nashville and San 
Diego. Two training teams led these sessions— both teams having prepared their 
presentations and sl^retegies together during the week of February 7. This joint 
preparation, combined with the use of a scripted Trainer's Manual , helped to maximize 
consistency within and between these two teams. 

Westat based interviewer training for the Pell Grant survey on a-.trairiing plan - 
which the company had developed and refined for other large-scale surveys. Training 
techniques included home study of a programmed text, interactive lectures to the 
entire group, and role playing (administration of a mock interview compiete with 
documentation). Central to "the ^training plan was the subdivision of the group of 
trainees into small groups of approximately 10 trainees who received individual 
attention from the' group leader. The field supervisors' functioned as group leaders 
during training so that they were able to observe and assist the interviewers they 
would later supervise. Additional trainers also acted as group leaders and reported the 
;p^^ at the end of each day of training, 

trainees who successfully; coiTipletjed interview, the final- exam, and who 

received positive evaluations from the group leader were given their initial 
assignments at training. ^ 

*.3 FIELD OPERATIOnI" - 

i 

In general, the field work went very .smoothly. Supervisors had scheduled 
conferences with; each, interviewer at least once a week, or more f recjpently if 
necessary. Similarly, the supervisors were scheduled to report to the field director at 
least weekly, although interaction between the home office and supervisors occurred 
almost daily. 

Assignments to interviewers were made by each supervisor on an "as needed" 
basis. Most interviews received new assignments weekly. The supervisors monitored 
the number of assignments being worked on by each interviewer so that no interviewer 
had a backlog of more than a week's work. Interviewers we're encouraged to schedule 
their appointments so that each week they would work a minimum, of 20 hours. They 
were also encouraged to schedule student interviews first, since the students were 
much more likely to become unavailable as the field period extended into May, when 
many institutions end their spring terms and begin summer vacation. 



In many cases, the interviewers had reasonably current addresses for the 
students that they were" assigned. Most were easily tracked through student 
directories or the local phone booki. . If a student could not be located through 
conventional me,ans, the interviewed deferred the atssignment bac|< to the supervisor. 
The superivsor then called the appropriate office in the instituliaoh, such as the Studefft 
Affairs Office, to acquire a current address. ■ ' 

interviews with students, interviewers occasionally were told by a 
student that they had never received a Pell Gr:ant. This is not surprising since students 
were sampled from both the recipient file: at the school and ^ the pending file of 
students who had been determined to be eligible &ut who had not, yet received a grant. 
Such cases were referred to the supefvisor> who called the financial aid administrator 
at the school to verify that the student was not a' Pjsll Grant recipient. ; 

Up to five telephone and three in-person attempts to obtain appbintment with 
each studenf or parent were required. Most appointments, however, were successfully 
made in fewer contact>attemptSv . - ' . / \ ' 

Although student resppnclents: were generally sampled 
institution, parent respondents v^ere scattered throughout the entire country. This 
meant that at some point, some interyiewers had to travel^o respondents located in 
far ^way places. Assignments of this typfe^W^ and Allowed to accumulate Until 

the last few weeks of the field period. Af .that point, supervisors coordinated 
interviewer travel plans with the field director, :and the interviewers were sent out to 
conduct the interviews. Because of the dispersion of these parent interviews, they 
were often conducted by an interviewer different than the one who conducted the 
corresponding student interivew. 

This procedure was not followed for 261 parents of independent students who 
lived distant from any interviewer. Documents for these interviews were shipped to 
the Westat T^elephone Center, where the interviews were completed. Since the 
interview with such parents was very short, we considered this to be a satisfactory 
way to reduce interviewing costs without jeopardizing the quality of the data 
collected. 




THE AUTOMATED SURVEY CONTROL SYSTEM (ASCS) 

An integral part of field management on the Pell Grant Study was the computer- 
assisted management system/ known as ASCS. The ASCS operated through small 
computer terminals located iy the supervispri' homes and connected through telephone 
lines td a computer. A similar terminal was located in Westat's home office for use by 
the field director. Each week the supervisors would enter * information on field 
progress into the system, and on a regular basis thfe terminal would print out summary 
reports on survey progress. The system was also used to transmit and receive, 
niessages to and from the home office, as well as among the regiorial supervisors. • . v: 



y Tbe ASCS generated^ thr^^e reports which were lisetf by the Regional Supervisors. 
ASCS Report //I, the Supervisor Interview Report, listed .the identification numbers of 
all cases currently assigned to an interviewer. Each week the system generated a new 
Report //I for each interviewer. This report served as a record of assignments and was 
discussed during the wee^|<ly supervisor/interviewer conference. ASCS Report^ also 
was generated weekly dnd presented production information on aiL4nteryiewers. It 
provided information such as response rate, hours and expenses per completed 
interview, and cost per completed interview. ASCS Report //5 presented totals 
showing the current disposition of all cases in a region. A review of this report 
provided an. accurate, overall picture of a region's progress in completing the survey. 

J ■■■■ " ^ " 

In addition to th^se reports, the ASCS was capable of generating 1^ more reports 
for use by home offic|L staff. These reports were used to monitor nationwide survey 
progress and provided detailed information about different respondent types (e.g., 
dependent students oJ parents of dependent students). 

' . \ * ' 

k.5 SECONDARY eIaTA COLLECTION 

Some of the ijems on the Pell Grant application that was filled out by each of 
the students in the Sample were verified using an independent source. The purpose of 
acquiring secondary^ data was to verify income and asset information with *'harder" 
documentation thar^ the self-reported income and asset data provided by the student or 
pare&it d^ng the I'lnterview. The major source of secondary^ data, used to verify 
income variables, was the Internal Revenue Service. Other sources were Financial 



ERIC 



i 



Institution Records (FIR), used to verify amounts deposited ,^Wihgs and ch^cking^ 
accounts; arid fax Assessor Records (TAR) used toV verify 'th^cl^*'>^alue> J^Z^^ 
percent subsample of homeowners. / _ V' -rv^ ' . - 
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The secondary data collection lagged behin^d the fiel^ int^^i^ihg by. several 
weeks, because the necessity ..;6f acquiring additional data, was* {^hl/ cieter mined 
through each. student or pi^rent interview. . / ' v ; . % V / * 

■■■■ ■ / • . - . iw*'^- ' 

/ 



*.5i 1 IRS Tax Retuhis , / , * ' 4 ' 



Westat rriade arrangements with IRS to acquire copies of each r(^s'pondent's tax 
returns for the 1981 tax year (for Ithose who filed a 10^0 or lO^OA) or a' notification 
from the IRS that a particular respondent's 1981 tax return was not found. If the IRS 
could not find a return, we assumed f\^x the respondent .probably did not file a 1981 
return. ^ 

The pirocedure specified by the IRS required that a release form be signed by 
each taxpayer authorizing Westat to obtain copies of the tax payer's 1981 return. IRS 
Form *506 is the release form used for that purpose. >^estat used a specially modified 
version of Form ^506 for the Pell Grant project. ^ ^ 

IRS Forms ^506 were sent to each of the students and parents in the sample prior 
to the interview, with instructions to fill out the form and return it to Westat. At the 
end of the interview, the interviewer asked the respondent to sign another Form ^506. * 
While this double procedure produced many duplicate release forms, jt^ ensured more 
com]5lete coverage than might be expected by using a single method. 

original Forms ^506 to the IRS service denters, which returned 
6,032 to Westat.' Of the 6,032 returned, usable copies of tax returns were obtained for 
^,007 survey respondents, the remainder (2,025) represented presumed non-filers for 
Avhom no returns could be found. 



'?.5.2 Financial Institution Records aTR)__^ t 

V During the in-person interview, respondents who reported having more than 
$^,000 in checking or savings acounts at the time of application for a Pell Grant were 
asked to sign a release form for each account so that Westat could obtain the balance 
on those accounts directly. (A small number of respondents who did jxot know the 
value of their accounts were also asked to sign releases.) As the releases were 
received at Westat, they were logged in and the date of application for each release 
was added to the forms. They were then sent to the .financial institution with a cover 
letter explairiing the study and a self-addressed envelope. Considerable telephone 
follow-up 'wa^ required to persuade institutions to reply. Of the 392 releases that were 
sent to financ^ial institutions, 365 were returned to Westat. 

*.5.3 Tax Assessor Records (TAR) 

For a. 25 percent sample of homeowners (among parents of dependent students 
and independent students who reported owning homes, at the time of application), the 
local tax asessor's office was cont^l^d->^ mail tg acquire home value. As with the 
FIRs, considerable telephone foUowup was required, both to identify the appropriate 
jurisdiction and to encourage a reply. Westat sent 365 forms to tax assessors; 36^ 
were returned. 'However, since some of these 36^ homeowners were laiter found to be 
ineligibile (either the student did not receive a grant or did not attend the school 
where sampled), 3^9 TAR^ were included in the data file. 



. ; DATA PROCESSING ^ , 

■ ' V ; .. ,: ,f ■ " ' ;■• / • 

This chapter e5;:plains riot only the automated data processing procedures used at 
Advance'd Technology and Westat, but also the premachine processing, that is, manual 
coding and editing. Quality control procedures for the required software are also 
discussed. 

5.i INTRODUCTION \ . 



Data for the project were derived from nine different sources, with different 
record and file structures and requiring differint amounts and levels of processing to 
be used for analysis and production of Jhe final report. "The nine data sources were: 



Student Questionnaires (SQ) 
Parent Questionnaires (PQ) 
Student Record Abstracts (SR A)" 
Institutional Interview Forms';or Questionnaires (IQ) 
Institution Control Group Forms (ICG) _f- ' . 

Computed Applicant RecoVds of^Student Aid R.ep6r|^:|SAR) 
Tax Assessor Records (TAR) 
Financial Institution Records (BIR) 
Internal Revenue Service 10^0/lO^OA Forins 



Westat, Inc., administered the Student and Parent Questionnaires, collected th^^ 
TAR, FIR, and IRS dita, and cleaned and edited all the;. data frorr^the^se^ 
Advanced Technplogy received both Westat's data and Westats edit sp^ctifi'catipns.^ 
The purpose of these reviews was' to assure that Westat quiality contrjbT:pr1x:edijres 
were followed and were effective. Adyancedc Technology monitored Westat's data 



collection schedule closely to ensure that no schedule delays were created that might 
have affected Advanced Technology's processing of the data. Upon receipt of data, a 
series of frequencies and descriptive statistics Were run in order to ensure data 
accuracy. This procedure served as a final step in the data editing process, and 
provided assistance in the following ar^eass \ 

^. • ■■ : • . 

• Identification of missed ^dit checks ** ;^ 

• Identification of missed data updates 

• Preparation of the final tabulations plan 

Advanced Technology prepared the specifications for frequencies and descriptive 
statistics using SAS. The remaining data (i.e., SRA, IQ, ICG) were processigdjaJine 
with the procedures outlined in this report. 

The plan addressed six groups of procedures: 

• Data control procedures 

• Coding procedures 

• Data entry procedures 

• Edit and update procedures 

• SAS merge procedures 

• Documentation procedures 

Each procedures group was composed of several steps or tasks which, when 
combined, produced a clean file data set for analysis. This process is depicted in the 
following figures. Figure 5-1, "Pell Grant Quality Control Project--Stage Three Data 
Processing Plan Data Flow" is an overview of the flow of data from collection through 
the production of a clean data file for analysis. Figure 5-2 "Pell Grant Quality Control 
Project— Merge and Analysis" shows how these data were combined to create a single 
file for analysis. ^ 
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• " FIGURE 5-2 

PELL graNj Quality (^ntrol prooect-merge and analysis 
ERIC • ' . 



5.2 INSTITUTIONAL DATA 



Advanced Technology collected two kinds of data during the institutional visits. 
Data about the institution and its financial aid administration procedures and 
packaging philosophy were the subject of the Institutional Questionnaire. The site 
visitors also examined the institution's records on -each student and ^completed an 
individual Student Recid Abstract. Site visitors mailed the IQ and associated SRAs 
to Advanced Technology^^roject office shortly after the end of each visit (after field 
editing). 



5.2.1 Data Control Procedures 

Data control is a critical *part of any data collection effort. In this particular 
application it became especially important because there were multiple sources of 
data dealing with the same respondent. M tightly controlled tracking instrument was 
necessary to monitor the processing ^st^tus of questionnaires and prevent duplicate 
entry of data. For these purposes, a series of data control logs were'established which 
indicated: 

Location of an institution 
Interviewer responsible for an institution 
Status of documents for a given institution 
Batch number for each institution 
Students within a given institution 

Logs for the SRA and IC<!l^3Dnsisted of three levels ^ djntrol. First was the 
Student Listing*^ Log (Fi^re 5-3), a computer-generated listing of Ml fsam^led students 
within each institution. As forms catne i^^from the fielrf, they ^Avere checked off 
against this log and any problems associat^l^ith either the institution or individual 
students were recorded. The Institution ^ntrdl Sheet was the second level of control 
(Figure 5-^). This form acted, as a control fS^m for institutions and provided summary 
statistics for each institution (e.g., number of documents for each institution sampled) 
and provided tracking of datct received, coded, and keypunched. ^The third leveLof 
contrai^as the Batch Control Sheets (Figure 5-5), which were attached to the front of 
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FIGURE 5-3 
STUDENT LISTING LOG 
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each keying batch to maintain counts and responsibility for assignments throughout the 
keying, editing, and updating. A copy of th$ Batch Control Sheet was also kept in a 
loo'Seleaf notebook so that the codirtg supervisor could track fotms from a central 
locatit>n at all times. 

The Student Listing Logs served both the SRAs and the ICGs. Therefore, there 
were no institutional cover sheets fgr batches of institutional questionnaires. 

All documents were received and batched according to institution. When the 
data preparation staff received a package from a sampled institution, they divided the 
contents into groups by form type (i-e., SRA, IQ, ICG). The coding supervisor recorded 
all forms in ^ their respective logs and determined coding assignments. When the 
volume of forms received for one day exceeded the coding capacity of the staff, forms 
were held in. at secure file to await coding assignment." After coding and all 
appropriate logging procedures, forms were placed into separate files to be batched 
for data entry. 

' Before going to data entry, forms for institutions were grouped* into keying 
batches. These batches then served as the quality control group throughout keying^ 
editing, and updating. Each batch was assigned a number and a Batch Control Sheet 
(Figure 5-5). A Batch Control Sheet was a listing of all institutions in a batch and 
included the information in the log book. These forms and procedures allowed precise 
-.^tracking of all survey instruments from receipt through coding, logging, editing, 
updating, and data retrieval. 

The construction of a l^urvey Control File also contributed to the plan for data 
control. Survey Control Files were developed for both the Institutional Questionnaire 
and the Student jRecord Abstract files. These files consisted of the identification 
number, the region code, and dependency status code for v^ach corresponding record in 
the various surveys. These files served ^ a further check on data accuracy by 
assuring that all ID numbers and accompanying identification information (i.e., region, 
dependency status) were properly keyed and that no duplication or extra cases 
occurred* This was accomplished by performing a merge tf the newly keyed data and 

the respective Survey Control File. . Any discrep.ancies in this merge f)rocess were 

■ • • ■ ' -V 

flagged and resolved. . . 



5.2.2 Coding EVocedures 

■ I ' ■ ■ , ■ 
. ■ ■ . ■ J ■ J , ■ 

Manual coding and cursory editing were tione under the close supervision of the 
coding supervisor. The coding supervisor's responsibilities included: 

. Ldgging inconiing fbrms ; ., : - 

Making coding assignments' ^ .. 

Monitoring coding production and r^ular reporting to the project director 

1 ' \ ■ ■ ^ ' " ■ ■ • ' 

iV^aintaining all logs and tracking f orms^ 

Assisting in error resolution , • <> 

■ ■ ' ■•'! ■ ■ ^ ' . .■. - ■ - 

t . Assisting in coder training 

• Maintaining lists of codes for open-ended questions 

The coding supervisor and the analysis staff conducted coder trairfing. Training 
consistediof an introduction to the study, the data collection instruments, and standard 
coding conV^rjitions and procedures. After the introduction, coders were instructed on^ 
the mechanic^ of coding every question on each of the forms (SRA, IQ, ICG). Coder 
training also included step-by-ste[^^structions in the keeping of logs, the procedures 
for resolving problerns in the data, and the system for updating information. 

The Coding Manual and Data Elements Dictionary contained iCompT ehertsive 
information abput all possible codes, acceptable ranges, and the coding conventions 
:'and procedures being used for the data collection. Each coder had a copy of the 
manual and was responsible for making ^1 of the necessary updates to his copy. Any- 
changes made to the coding manual were made during the daily meeting of the coding 
staff. Since the manual was the* primary reference for any questions of coding and 
code definitions, it was imperative that each coder's manual be kept up to date and 
accurate. Any changes had to be added to the , manual, dated> and initialed by the, 
coding supervisor^ The coding staff met daily to update the "data elements dictionary 
and to ensure that new procedures and codes were properly implemented. 

During the training sessions all coders were instructed in^the coding and editing 
procedures for use with each of the data collection instruments; however two coders 



were assigned primarily to the coding arid editing of the IQ and the ICG. Since all of 
the Coders were trained to code all three forms, coder time was reallocated as needed 
to meet changes in worlifload or time pressures that developed during the course of the 
.data collection. , , " 

Problems encoCintered during the coding process were documented and resolved. 
The coders brought all problems of inconsistency within a questionnaire, responses out 
of acceptable ranges, illegible responses, need for new codes, or incomplete responses 
to the attention of the coding supervisor. The form in -question was set aside until t^e 
problern had befen resolved. When problems arose with the data that required a higher 
rauthority for resolution, th^-^coding supervisor brought them to the attention of the 
manager of data analy$is oi? the technical director^ 

The standardized coding conventions employed-^n all three form^ inlcuded the 
following: * 

All numeric entries were rightrjustified and zero-filled. 

A code of "99*J was used to indicate that a question was ask^d but no 
response was given. - 'V; ' 

A code of "97" always indicated that a question was a.sl<ed but the 
^ respondent was uncertain or could not determine the answer. 



; ■ - - . ■ . , p , . ■ 

Blank spaces on the form indicated that a question was not asked due to a 
skip pattern. ; . 

All changes, corrections, and updates were made in red ink. 

. Pill forms received were opened, logged in, and filed in a secured room^ All 
coding and editing were done in an office ^adjacent to the file room. Restricting the 
receipt of forms, codingSperations, and filing to two rooms limited the .possibility that 
any data woulS be Ipst xx^ispl^^ arid assumed tlfe security and cbnfidentis^lity of the 



data. 



In., addition to the detail^ coding specifications described above, we devised a 
graphic Representation of all 'Sfta records , in the form of data record layouts, to 
accompany each codebook. These record layouts provide a graphifc representation of 
the physical position of all of the data elements in a given record as well as logical 



groupings of the va^rious Idata elements. Thj^^^a^ ii;i..the 
development of the comprehensive logic checki^<3tr the editing process. The data 
definitions for all programs accessing these data were derived from record layouts. In^ 
the event that OSFA ever requires a documented OS file, record layouts will pirov^ an 
invaluable resource in preparing documentation. - / ' 

5.2.3 Data Entry Procedures 

During the production phase of data collection, erttry .and editing, we sampled a 
smaill group of questionnaires to serve as a quality control, check on the data entry 
vendor. On this sample we compared keyed data from data entry with hardcopy 
documents to estimate the keying error rate. Errors could be the result of 
misinterpretation of the interviewer's marks arid might not be caught during 
verification. This sample quality control check found no data entry errors by the data 
entry vendor. * 

We described all edit ch'egks in ^ Computer Edit Instruction Manuak This manual 
had two purposes: lOj to provide detailed specif ication , to the programme-r for the. 
prepaption of the edit programs; 2) to serve as a tpol in err©r resol(itioh prior toethe 
update cycle. Based on the coding scheme develbped ^f<3r the yi^rious .codebooks, a 
comprehensive series ,.Qf range checks was applied to ali variables in eo^th survey ; 
instrument. Range checks tested far all valid codes and for ml|sing,CQ(les4"' • ^ 



We employedB* cdnsistent set of* coding contentions for aH the, various survey 
instrument? to aid in coding and the resolution of errors. Missing value codes wef^ the 
same for, all questionnaires and, ^i^henever possible, we assigned the same cb^le to 
applicable valid responses which could be given to several questions. 

To identify all applicable logic tes^Sj we developed a data flow diagram for each 
survey instrument. This flow chart illustrated the various "skip patterns lhat appflj||ired 
in each questionnaire and aided in identification of significant relationships between* 
questions. Th addition to those skip pattern checks, we made che^s for all apparently 
inconsistent responses. Inconsistent or contradictory information in the data wa^ 
detected by the logic check .module and was reviewed by an analyst. . , 



. The coding supervisor and systems analyst developed range and logic tests that 
^ere reviewed by the manager for data analysis. After a draft of all checks had been 
prepared, the entire data processing team performed a structured walkthrough of 
them. This walkthrough' helped to identify any additional checks needed to eliminate 
redundancy, -and to minimize the possibility of error from the interviewers, coders, or 
the data entry vendor. ' , 



The edit programs produced error reports which were forwarded tS the' coding 
supervisor for error resolution." ' The coding supervisor, using the Computeh Edit 
Instruction Manual, compared error reports with original questionnaires -and resolved 
the problem using the predetermined error resoluti6rj;;itgchniques. - the resolved edi;t 
repor^t was forwarded to coders to prepare an updateJra'nsaGtion which was keyed into 
an ALPHA transaction file. This, transacfibn file was processed by the ujidate 
program, which posted it against, .^he master record. - The upciate program, also 
produced a report detailing all transaction records, additions, deletions, ;and 
modifications (displaying the record both before and after modification) with summary 
statistics. The data processing procedure included the use of a batch update p^;.ogram. 
The update program, like the edit program, was written in COBOL for ease of use and 
maintenance. There were severah^iistin(^ process for the 

update cyQle: ■ . ■ '■ ^ ■ ■ , - . 

, ■ . . ' ■ ' ■ . . ■ - 

•• Greater quality contPii^^as achieved since each' update transaction had a 
printed record for auditing. 

• Coders spent less time on line. They ' were abl^ to enter transaction 

records without searching the files. ' . ■ -v • 

■ , . ■ ■ .■ 

• - Each update pass wrote a /new file so that, the old version with the 

transaction file served as a backup. ' ' ' 

• Access t9«='tt1^aw data was limited to the batch cycle under the control o-f 
/ _ the coding supervisor, assuring greater data security. / :^ 

'• ■ • 

The file management structure required that static . batches (those not yet 
pronounced "clean") remain on disk as' members of a partitioned . data set (PDS) to be 
updated by the batch cycle. As each batch was pronounced clean, it was transferred 
to tape to await concatenation with other clean batches. This provided enough 
efficiency to allow the use of traditionally scarce disk space at COMNET. 



Backups consisted oi old members of the static PDS. In the event- that they 
could not be used, the originaf da^'a transmitted from the data entry vendor wof^ 
available. Since there was an audit trail of update transactions, it was a i^l^ttvely 
simple matter to reconstruct the most current data from the original. 

\ 

5.2.* SAS Merge Procedures 

After the data had been edited and appropriate updates had been made, the <;iata 
(SRA, IQ) were combined with other data from Westat (Parent, Student, IRS, TAR, and 
FIR Data) to create the analysis file. This file was created through a complex series 
of inerge^ performed in SAS. Westat prepared a data, elemerj.r. dictionary of the Stage. 
.One files, which the data processing and analysis .staff revieVed^ However, the staff 
decided that new narning conventions would be more useful.- ^ i ' 

To produce an accurate analysis file using new code, the programming team and 
the^-analysts reviewed the "best v^dlue" selection process to develop a more efficient 
fneans of producing the analysis file. To aid in this review process, a new systems flow 
chart detailing the Stage One merge was developed. This^flow chart helped the team 
identify points at which efficiency could be enhanced. 

The use of SAS for the merging of the Pell Grant data offered significant 
advantages because of power and flexibility. However, caution had to be employed 
when merging datasets to insure that the desired values were incorporated* The 
mergicig of multiple files with many shared, similar, and discrete variables is 
vulnerable to error in several ways. Files do not always merge exactly as planned. 
Sometimes variables, or even entire records, are lost without a trace because of 
insufficient verification of merged datasets. For this reason, reports were produced to 
present in detail those cases that did and did not merge in order to verify the accuracy 
of the software. SAS has excellent self-documenting abilities in the input and output 
process and provide^ detailed information as to what data entered a step, what bdta 
exited a step, and what happened to the data in the process. The programming and 
analysis staft rnonitored this trail of information closely at crucial points to ensure the 
correct execution of the program as intended. 



5-1^: 



The creation of the analysis was .logically dtWded into two procedures: 1) a 
range process where all sources of data were brought together on t|Tie basis of common 
keys; and 2) the selection of the "best" documented value frqp available data. J^is 



process is depicted in Figure 3-6, ^ 



The first step in the merge process was to reformat the IRS data, ^cause of* its 
hierarchical file structure, the IRS data could not be used without .reformatting. 
Therefore, we created a flat file containing the specific variables needed for analysis. 
This reformatted IRS file was then combined with data from the Sttldent -Questionnaire 
(SQ), Parent Questionnaire (PQ), and Financial Institution Records (FIR)t 'Since ^lestat^ 
had collected all of these data, the merge was performed by Westat identification 
code. * ^ ^ 

■ t 

Our next step was to combine the data from the Student Record Abstraf:ts (SRA) 
with the data from^he file of. Computed Applicant Records (CAR). Because the CAR 
file..did not contain Westat's identification code, we performed this match u|^g the 
student's social security number.^i The output of that m^rge (BGP.BQC.SRACAR) was 
then merged with the output of the previous merge (BGP.BQC.FIR.IRS.PQSQ) along 
with the Tax Assessor Records (TAR) to create the first composite file 
(BGP.BQC.ALL.SEVEN.DATASETS). We executed two additional programs after the 
creation of this composite file and before the best values selek^tion. The first of these 
programs updated the file with SAR data on ADS applicant|. The second program 
added information from the Pell Recipient History Master Filf on Section 3 of the 
SAR for institutions that exercised .their option to submit these data on tape. We also 
performed several updates on the co-mposite file to assXire clean and accurate data. 
These began with BGP.BQC.ALL.SEVEN.NUMBER2 and '"ended with 
BGP.BQC.ALL.SEVEN.NUMBERS. This final composite fUe consisted of 3,791' cases 
that were used for best value selection. • v 

The first variables that were computed for best value i^lection were those that 
were common to all students, regardless 0% their dependency Status. A major purpose 
of these, which is explained in more detail in Volume 1, was to determine the' "best" 
dependency status (BGP.BQC.INDDEP.BESTOl). If the best dependency status was 
"independent" we ^executed the program to determine best values on all application 
items for mdependent students (BGP.BQC.INDVARS.BE5T0I). If the besf^dependency 
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FIGURE 5-6 

DATA BASE MERGE AND BEST VALUE SELECTION PROCESSES 
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FIGURE 5-6 (Continued) / 



DATA BASE MERGE AND BEST VALUE SELECTION PROCESSES 
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status was "dependent" we executed two proerams: one tj!> determinie best values for ' 
parental application iiems (BGP.BQC.PVAITS.^E-SJOl)' felnd one to determine best 
values for dependent student application it«^ (BfeP.BQC.DEPVARS.&ESTOl)/ The 
merger of these best values files was a com|)dsjte analysis fije'(BGP.BQC.B^ 

used in the analysis of award errors In Volume T. • ^ • , . 

■ . ■• . / ■ ■ ■ ^ ■ '■ , " '. • ■■■ • '■ • 

. . ■ : ■ ■ . ^ . , . ■ ■ ' • 

5.2*5 Documervfation ' ^ • ^ \ 

■ ■ ■ .■ . . ■ / ■■ , 

I. : A Dataset Documentation Form (Figure 5-7) \)i/as liyed out for all f ^.les to provide 
a more complete documentation of data files*knci* program ^iles. Jhis for^tn included 
'detailed physicarand logical characteristics of ^11 iil^s, whetj:ier they wer^ or 
arid , whether they resided on disk or tape. 'Cpmptete l|?jings o^all datas<^;^- were ' 
^ nnaintainecl in a, looseleaf notebook; These' foi^rtis 'provided d^ftailed tra^fking 
'■ mechanism and aided in the final documentation .rff ^al^i^ets'.^^ \^ '^ijc^y'C - 

. SAS merge procedures . were also dodiument^ ^^ti^rnalJM* andy -extet^allVVi 
throughout the pi-o.cess. External documentation con^sl?ed|pa detaSed description 
the merge cycle to be recorded on a" form, one copy bei^g^fcbjiinjjl with computer 
output, the other copy bein^ kept in a looseleaf notebook. P^is documentetio^r: 
provided an opportunity to describe in detail any prol^l^tSn^ e?ict)ilMter^ arid p^ovidfiiC. 
the foundation of the final report detailing the merge |i||rocfeJi%u the internal 
documentation consisted of a brief description of the inputs, output^*^ and proces/^f 
each SAS program, and appeared at the top of the^soui^ li^in^ as l^ s^jction of " 
comments. ^ ^ * /v -^T ' 

' 5.2.6 Statistical Analysis s *^ s 

After editing and updating the data and creating the analysis ^fij^e, we began tt 
* statistical analysis. The production of all tiables was programmed SAS, i>ased'^ 

specifications provided by the manager of data analysis, who coordinated all Jftn|^tic ' 
\ Activities and was the primary source of requests for production^ <5f {aj^s. The® 

specifications for analysis included the following: ' 

# Purpose (place in report and associated research question) \ 

• Input files . * 
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datXset documentation form 



Population (which cases are included or exclud^ 
^nalytic specifications "t 

Output configuration * " ' 

.< 

Quality control output (tables designed to serve as a check on the 
programming) 

Output files and retention-S ■ 

5.2.7 Software Quality Control - . ' 

Assuring the quality of software must be a. prime consideration of any data 
processing effort. In order to maintain its commitment to the quality of the software, 
Advanced Technology adhered strictly to the following procedures for program 
development for the Pell Grant Quality Control Study: 



Structured walkthrough of edit program specification 

Structured walkthrough ol merge and "best value" selection process 

Structured walkthrough of award programs 

Review of ad hoc analysis specifications by the manager of data processing 
and quality control analyst a ^ 

Manually produced test' data to verifv edit, ua|iite, merge, and award 
programs ' ^ 

Ten percent sample of live data to te5t programs 



Production of marginal tables for all variables " 

^ ' *^ ^ 

The flow of this process can be viewed in Figure 5-8, Program Development. 
Once specifications for a program were developed, the staff member responsible 
reviewed those specifications with the quality control team consisting of the manager 
of data analysis, the manager of data proliess^ng, and the quality control analyst. The 
team determined whether the specifications met the following qualifications: 

• Accuracy— are the appropri^^ inputs specified? 

5' ■ ■ ' ■ 

• Completeness— are all necessary, variaibles, transformations, and subsets 
specified? ^ ; 

' * f , ' A . ■ ■ 
....... 1^:. 
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REVISE 



REVISE 




Efficiency— are all the specifications necessary? 



If the specifications passed these tests, they* were assigned to a programmer for 
development. Once specifications were approved, and a program was developed, a 

similar process was followed to ensure proper programrning. 

■ * »■ 

The program was then thoroughly tested by first applying it to a batch of hand- 
calculated test data. The resulting output was compared to hand-calculated results to 
verify accuracy. The second step in testing programs was to run them on a 10 percent 
sample of live data. This test performed an additional check on the program to see 
that probable results were obtained (i.e., whether the resulting analysis fell within pre- 
established acceptable limits). In each of these test steps, if errors were detected, 
they were corrected promptly and re-tested until the program was judged acceptable. 

Some data analysis was performed on an a3 hoc basis. Such analysis usually 
required quick response and received a technicdl review by the manager of data 
processing and the quality control analyst. These reviews were less '^formal than the 
reviews performed on the edit or merge programs, but no less thorough. They 
examined required inputs, desired outputs, and necessary manipulation of data. Before 
implementation, all ad hoc programs were also put through the same testing 

procedures used oh production programs. 

' ■ ' ■■ . ^ ■ ' 

,1 

The testing process was designed to follow all possible logic paths and test all 
bpundary conditions of the software. In order* to achieve this. Advanced Technology 
produced d^package of test data that included-fex^ accurate and inaccurate data. 

*. ' ' ■ ^ 

5.3 INDIVIDUAL DATA 



We^tat wais responsible for the collection and progressing of all individual-level 
data except fof Student Record Abstracts, including student and parent intervie^ and 
data about individual financial status from tax assessors, financial institutiorWjand 
IRS. 4 

- The processing for each of the five independently constructed data files 
associated with the survey (Parent Questionnaire, Student Questionnaire, IRS Returns, 

ERIC • . ^ 



Financial Institution Records, and Tax Assessor Records) was done using very similar 
procedures. . Data processing . for . the student and parent data was, however, 
considerably more complex and\ time-consuming. The work required at each step in 
processing the ^tudent/parent data was commensurately more complex and time- 
consuming. 

5.3.1 Receipt Control 

Receipt control, using computer-generated logs of all materials acquired relating 
to the student/parent sample, was. the control and monitoring point for all materials 
sent to the field and returned to Westat. The master receipt control log listed the 
study i(^entification number, name, address, and Social Security number of each 
student/parent pair. The log was organized in tumeric order by student identifier 
within school. Space was provided ^for recording updated information on names and 
addresses, the date each questionnaire was receiyed from the field, the completion 
status of the document, and the coding batch number assigned to the document. 
Labeled columns were also provided for recording the receipt of IRS Release Forms by 
mail (from the initial information mailing) and from inside the questionnaires (obtained 
during the interview). When IRS lO^fO and lO^OA Forms were received from the IRS 
Service Centers, they were coded with case identifiers and entered in the master log, 
with the IRS coding batch number. 

Westat made an additional check during the log-in procedure for/the IRS returns. 
The receipt control clerk manually checked the list of dependents listed on the 
parents' return, and identified the student among them. N 



5.3.2 Coding 

Coding manuals were prepared for each of the five data source^ documents used 
in the Pell Grant study for ^se in training the data preparation staff, serving as' a 
complete and detailed reference for analysts, programmers, and data preparation staff 
and providing documentation for the Pell Grant study data files.. Each coding manual 
consisted of an introduction to the study procedures and pu?]|^bses,, a review of general 
data prepara1;ion procedures to be followed, and coding and editing specifications for 
the five data sets. 
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We^tat selected about 30 survey processing personnel for training as coding and 
editing staff for the Pell Grant .study, including two group leaders, who were selected 
based on their skills and qualifications. The group leaders were assigned as assistants 
to the coding supervisors and as code verifiers. As much a§ possible^ Westat chose 
experienced survey processing personnel as* coders for this project to minimize the ■ 
amount of training necessary on basic coding sRlUs. • f 

Coders were trained in groups of between ^^nd 10. ' Each group was trained to 
code on one of the five study data sets. Betweep/1 and 8 hours of training time were 
required, depending on which data set was to be coded. 



Questionnaires were precolumned before printing so that coding could be written 
directly on the questionnaires. FIRs, TARs and IRS 10^0 and lO^OA Forms could nqt 
be precolumned, so transcription sheets were designed for the coding. , ' , 

^ Coders were assigned v/ork by coding batch and were required to complete the 
^ coding of one batch before beginning work on another. Errors found during 
"^verification by tH^^ervisor were first noted in a coding error log, then discussed 
witii the coder committing them. If persistent|errors were 'discovered, a coder was 
asked to review previous batches and correct t}>em. Problems found during coding but,^ 
not resolved in the codirijg specifications were.ddcumented a;id referred to ^ supervisor 
to.>be^ resolved. PacticuVrly <iiffacult cas^w referred to a weekly meeting of 
senior project staff for resolj'' ^ 




. O^:pasional problems with* i|!egible figifres arose in.tfje coding of photocopies of ' 
IRS tax forms. It was, sojjn^etim^s. necessary in' t^ese situ^tiohs ^tdj code- illegible data 
elemerits as . missing valines, ^jfe taxpayers do not completely fill out the 10^0 A 
Form when, filing eij^ttising •their op|i6n to 'We^, the IRS calculate their taxes. These 
lO^OA Forms are bl^nk'±)eli>w line 12.' wistat coders were trained to fill in the missing 
items on these blank forms using a \3%\ tax table. J . ^ ^ '^^ 

Thez-major codrng ^problem for the Studfent and Parent Questionnaires wa^ the 
large number o| questions in each questionnaireAwhich were open-etided. It Wias^ not' ' 
possible prior to tHfe beginni^of codin^^ to djsv^^ lists of all the possible pespopses to 
\these items. !$ince this proble^n wa^s expected, ^^^^ system of dealing r 



was implemented at the beginning of coding: responses which were not on the 
predetermined list of codes in the coding specif ications'were documented and referred 
to the supervisoris who constructed codes for the new items. New codes w^ce 
distributed each morning on a Coding Change Sheet, Coders^were responsible for 
keeping their manuals up^ to date and vC^ere required to record each issue of the coding^' 
changes in a log. ^ ' ^ : \ 

5,^3.3 Data Retrieval o 

. Westat trained its coders to ' edit the data croUection instrument during the 

coding* Editing involved checking for \readabilit3j^, sensibi and/ following of skip 

patterns. (Editing was much more important in the coding of the ^questionnaires than 

in the coding of the secondary data, sources.) A general rule Was established that all 

primary verification questions in the questionnaire must have codable responses.. When 

. .:o:' / . ' . . ' ' ■ fS'' 

coders found errorleous skips, illegible answers, or illogical responses in* any of the 

verification questions, they docjtimented the problem and referred 'the case to a 

supervisor for data retrieval. ' V: , 

Experienced, specially: trained telephone interviewers rieved ' doubtful or^ 
missing data. Case problems were described on a Data Retrieval Request ^orm .which 
also served as a record of calls for the interviewer. ^;Data retrieval was attempted on 
367 Parent Questionnaires and ^07 Student Questionnaires. In addition to data 
retrieval due to* problems found in the coding edit, (fata retrieval requests were also 
generated during machine editing. ^ ' \ 

'5«3.^ Key Entry apd Machine Editing r 

. *' ' . • . • ^ 

« ■ ■ ' ' ' ? 

Coded docurnents and questionnaires which had bejsn verified were taken to the 
Westat data entry staff ' in groups (called "keying .batches") of approximately 100 
docqments. Coded documents' were keyed into an in-house disk storagje ^ystem^ and 
then key verified from the disk. After keying and key verifying, the data were 
transmitted to Westat's VAX computer where they were stored on tapes to await 
machine editing. • - / , - . ^ 

' • ' ■ ■' ' '■• ' ^ ' ■ ■ \i ' 
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All data sets except those for the Tax Assessor Records and Financial Institution 
Records were machine edited with special purpose COBOL programs, written to check 
for out-of-range codes, incorrect skip patterns, and inconsistent response patterns. 
The data sets were grouped by keying batch for editing. 

Machine edit staff were trained coders, the majority of whom had earlier coded 
and edited the questionnaires as they were sent from the field. The coding supervisor, 
who had had extensive machine editing experience, also supervised the machine editing 
of the questionnaire files. 

Machine edit clerks wrote file updating instructions on transcription sheets, 
supervisors checked these instructions and sent them to the data entry office for 
keying and transmittal to the computer center. Updates were made to the files by a 
special purpose COBOL update program. After each updating was complete, another 
editing cycle was run to verify that corrections had been made and to check for new 
errors. The update-edit cycle was repeated until each batch of data in the data set 
was clean. 

Frequency distributions were run on all variables in each data set after the 
machine edit process- was complete. The frequency distributions were carefully 
checked to uncover any inconsistencies .and errors not found in the machine edit 
process. Any identified problems were corrected and new frequency distributions were 
run to assure accuracy. 
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DATA ANALYSIS 



In the last ch apter we discussed the processing of the data and the merger of all 
recipient-based data into a single file. This prepared the data for analysis, which 
itself was a complex Process. Included in this chapter are descriptions of best value 
selection, detailed resejr :h questions, and table production and statistical analysis. 

6. 1 BEST VALUE SELECTION 

The merged recipient data base is a compilation of cleaned and edited data from 
seven different data sources: Student Record Abstracts (SRA), Student Questionnaires 
(SQ), Parent Questionnaires (PQ), IRS Trx Returns (IRS), Financial Institution Records 
(FIR), Tax Assessor Records (TAR), and Computed Applicant Records (CAR) of the 
Student Aid Report (SAR). This merged data base, however, includes possibly 
different responses to identical items. For any one recipient responses to questions 
regarding Adjusted Gross Income, for example, might be contained in the SRA, PQ, 
SQ, or IRS. Thus, until further compilation of this data was performed, this raw data 
file was of limited use in analysis. 

The basic analytic task of the study was to determine the difference between the 
"best'* value for a given item and the value that appeared on the application. Any 
discrepancy for a student or parent item might have affected the Student Aid Index 
and thus the amount of the award. Discrepancies in application item values that had 
payment consequences contributed to program-wide estimates of efror. But, to 
determine the existence of a discrepancy required the selection of one value from all 
of the possible sources as the best value. This was accomplished according to a 
detailed process of best value selection. 
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6.1.1 Strength Of Data Sources and Dcxiumentation 

The process of best value selection was a combination of objective and 
subjective judgements about the relative strength of each possible source for a given 
application item. These were based on the views of Advanced Technology's senior 
staff, which included acknowledged experts in student financial aid and experienced 
analysts, with the corroboration, of OSFA project staff and an outside consultant. The 
process was highly detailed and, individually tailored for each application item. 
Decisions for best value selection were based jointly on the source of the data and the 
supporting documentation provided. Across data sources, there was a general 
hierarchy of strength which was followed. An external, reliable data source (IRS, FIR, 
TAR) was always considered first. If there was no such source present and 
documentation from other sources was of similar strength, the order followed was PQ 
or SQ (as applicable; for dependency status items where both PQ and SQ might 
contribute, the PQ preceeded the SQ), SRA, and, if no other source was present, the 
application value from the CAR. 

For supporting documentation, a separate hierarchy was established for each 
application item. In general, values documented by copies of directly relevant 
external records were first, such as a copy of a recent real estate appraisal for house 
value. These were followed by values with documentation from other external sources 
deemed to be less directly relevant or timely, such as a copy of a document showing 
property insurance held on a home, or a two year old bill-of-sale on a home. The next 
level of documentation was a statement from a knowledgeable professional (such as an 
accountant), followed by personal records from the respondent or a notarized 
statement from the respondent. Some documentation was considered unacceptable 
and was never used, since it was considered less reliable than the application. An 
example would be data retrieved by telephone or an irrelevant item from a tax return. 

When the general hierarchies for source and documentation are combined it is 
easy to see the complexity of the best value . selection process. This is illustrated in 
Figures 6-1, 6-2, and 6-3 which show the priorities used for Adjusted Gross Income (for 
independent students). Home Value (for parents^ of dependent students), and Supported 
by Parents, 1981. The first two figures provide different examples of items for which 
documentation is often preser^t, the third figure illustrates the subtleties involved in 
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FIGURE 6-1 

BEST VALUE SELECTION PRIORITIES^ 
ADJUSTED GROSS INCOME FOR INDEPENDENT STUDENTS 
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FIGURE 6-1 (Continued) 

BEST VALUE SELECTION PRIORITIES: 
ADJUSTED GROSS INCOME FOR INDEPENDENT STUDENTS 



12 State Tax Return- 
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17 IRS receipt/Treasury Dept. 
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15 Separated student's portion 
of 1040 
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minus CW-S 

15 Statement from social agency 
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FIGURE 6-1 (Continued) 

BEST VALUE SELECTION PRIORITIES: 
ADJUSTED GROSS INCOME FOR INDEPENDENT STUDENTS 
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FIGURE 6-2 (Continued) 

BEST VALUE SELECTION PRIORITIES: 
HOME VALUE FOR PARENT(S) OF DEPENDENT STUDENTS 
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FIGURE 6-3 

Cd BEST VALUE SELECTION PRIORITIES: i 

■ SUPPORT BY PARENTS 1981 FOR ALL STUDENTS 



an item for which little documentation is typically available but a best value must be 
determined. Note that in the absence of documentation in Figure 6-3, agreement from 
multiple undocumented sources is considered strongest. 

Best value selection must follow a closely prescribed order since the best values 
for some items were needed to determine the best value of other items. Consider, for 
example, that one must know the best dependency status, marital status, and tax filing 
status in order to select the best Adjusted Gross Income. (If the best dependency 
status is dependent, and the parents are separated but filed© joint tax return for 1981, 
that tax return cannot be used for AGI without additional documentation regarding the 
proportion of income received by the supporting parent.) 

A list of all items for which best value priorities were developed appears in 
Figure 6-^. Since separate priorities were sometimes required by dependency status, 
these are listed by dependency status. 

6.1.2 Premises in Best Value Selection 

The selection of the best value from among several competing sources is limited 
by certain inherent restrictions in the nature of the Pell Quality Control study. Our 
aim is to verify the application values against the best sources at our disposal. Thus, 
our best value, even if it is from the highest priority source, is only best relative to 
others available. We cannot investigate the v**racity of the sources we use. Thus, an 
IRS-certified copy of a tax return is our best source for several items, based on the 
assumption that the tax return is accurate. If that tax return was itself inaccurate, 
for whatever reason, our best value would be inaccurate. 

A basic premise of best value selection is that application values be changed only 
when there is evidence of a value from a better source. Thus, in the absence of 
documentation we presume that the application value is correct. This presumption 
may occasionally lead to the rejection of a more accurate but undocumented value and 
the underestimation of absolute error. However, it is based on the simple and logical 
assertion that accurate recall of undocumented values was better at the time of 
application, when the item in question was current, than it would be more than one 
year later. 
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ALL STUDENTS 



Determining Student's Dependency Status 

Expected Social Security Educational Benefits 

Expected VA Educational Benefits 

Marital Status 

Support By Parents 1981 

Support By Parents 1982 

Claimed By Parents 1981 

Claimed By Parents 1982 

Lived With Parents 1981 

Lived With Parents 1982 

Child Support 

Other Welfare 

Non-Educational VA Benefits 



Untaxed Unemployment Compensation 
Interest on Tax Free Bonds 
Untaxed Pensions and Capital Gain^ 
Untaxed Housing Allowance 
Earnings Not Reported on Tax Retui^^ 
Interest/Dividend Exclusion 
Any Other Income 
IRS Filing Status 

Applicant's Expected Taxable Summer Income 
Applicant's Expected 'taxable School Year Income 
Spouse's Expected Taxable Summer ^MCome 
Spouse's Expected Taxable School V^^f" Incorne 



INDEPENDENT 

Filed/Did Not File Tax Return 

Adjusted Gross Income 

U.S. Taxes Paid 

State /Local Taxes Paid 

Applicant's Income 

Spouse's Income 

AFDC 

Other Income/Benefits Total 
Number of Exemptions 
Household Size 

Number in Postsecondary Education 
Medical/Dental Expenses 
Elementary/Secondary Tuition 
Cash/Savings/Checking 



STUDENTS ONLY \ 

Purchase Price of Hpme 
No Home Owned 
Home Value 
Home Debt 
Investment Value 
Investment Debt 
Farm Value 
Business Value 
Business/Farm Value 
Business Debt 
Farm Debt 
Business/Farm Debt 
Itemized Deductions 
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DEPENDENT STUDENTS ONLY 



Filed/Did Not File Tax Return 
Adjusted Gross Income 
U.S. Taxes Paid 
State/Local Taxes Paid 
AFDC 

Other Income/Benefits Total 
Cash/Savings/Checking 
Purchase Price of Home 
No Home Owned 
Home Value 
Home Debt 

Real Estate/Investment Value 



Real Estate/Investment Debt 
Farm Value 
Business Value 
Business/Farm Value 
Farm Debt 
Business Debt 
Business/Farm Debt 

Dependent Student's (and Spouse's) Net Income 
Dependent Student's (and Spouse'>) Net Assets 
Applicant's Income 
Spouse's Income 



PARENTCS) OF 

Filed/Did Not File Tax Return 
IRS Filing Status 
Adjusted Gross Income 
U.S. Taxes Paid 
State/Local Taxes Paid 
Father's Income 
Mother's Income 
AFDC 

Child Support 
Other Welfare 

Non-Educational VA Benefits 
Untaxed Unemployment Compensation 
Interest on Tax Free Bonds 
Untaxed Pensions and Capital Gains 
Untaxed Housing Allowa'ice 
Earnings Not Reported on Tax Return 
Interest/Dividend Exclusion 
Any Other Income 
Other Income/Benefits Total 
Number of Exemptions 



DEPENDENT STUDENTS 

Household Size 

Number in Postsecondary Education 
Medical/Dental Expenses 
Elementary/Secondary Tuition 
Cash/SavLngs/Checking 
I\irchase Price of Home 
No Home Owned 
Home Value 
Home Debt 

Real Estate/Investment Value 
Real Estate/Investment Debt 
Farm Value 
Business Value 
Business/Farm Value 
Business Debt 
Farm Debt 
Business/Farm Debt 
Marital Status 

Parent's Social Security Benefits 
Itemized Deductions 



FIGURE 6-.* (Continued) 
LIST OF ITEMS WITH BEST VALUE SELECTION PRIORITIES 
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There were a few exceptions to this premise of not accepting undocumented 
values as best values. These were applied systematically to items which were 
composite totals on the application for which the separate parts were not reported to 
the central processor. The most important of these was Other Nontaxable Income. 
Individual contributing elements to Other Nontaxable Income, such as child support or 
untaxed unemployment compensation, were recorded on the application worksheet, but 
only the total was reported on the application itself. Thus, our efforts to select the 
best value based on documentation of the separate parts could not proceed unless a 
provision was made for a default value to be used when no documentation was 
available. Recall that individually reported items used the application value as a 
default value. When a person reported during the interview the receipt of $2500 in 
undocumented child support we could not use a value of zero without grossly 
underreporting income. Nor was there any alternative value to be picked up from the 
application, since the application value included nontaxable income from all sources. 
Thus, the only reasonable approach was to accept the undocumented report. 

Some other items also required the acceptance of undocumented values because 
the separate contributing parts were not reported on the application. These were the 
contributing parts to Dependent Student's (and Spouse's) Net Income and Dependent 
Student's (and Spouse's) Net Assets. In addition, to avoid large overestimations of net 
worth, we accepted undocumented values of Home Debt, Real Estate/Investment 
Debt, Business Debt, or Farm Debt when corresponding documented values for Home 
Value, Real Estate/Investment Value, Business Value, or Farm Value were present. 

6.2 DETAILED RESEARCH QUESTIONS 

Given the amount of data collected it was necessary to determine in advance 
those areas that would be the focus of data analysis. While all analysis of error 
followed the general equations set forth in the Appendix of Volume 1, the number of 
specific error items and comparisons between items was so large that potential 
research questions were delineated in advance and submitted to the project officer 
before analysis began. A total of 1^5 research questions were prepared, divided by 
topics likely to be included as chapters in Volume 1. These specific research questions 
presented rates and amounts of error (e.g.. What is the marginal impact of application 
item error on net student error?), bivariate interactions or crosstabulations of error 
with other variables (Does institutional error differ by type and control of 



6-12 



EKLC 




institution?), and multivari^-te analyses (What are the contributing factors of student 
error?). For each specific ^JtJ^stion the measures that would be used were delineated; 
for bivariate and multivariate 3.nal/ses the independent and dependent variables were 
listed separately. 

These research questi^n^ served as a guideline for the subsequent analysis and 
the production of tables to^ findings. Based on the initial results of the . early 
analyses, specific researQt^ '^u^stions were revised, expanded, or, in some cases, 
dropped. Together these r^5Garch questions enabled us to pursue data analysis in an 
organized fashion. Each of ihe tables presented in Volume 1 represents the findings of 
one or more specific resear^^h questions. 

6.3 STATISTICAL ANAL/SIS AND TABLE PRODUCTION 

Before statistical anaA3'Sis of the data began, a rr^aster analysis file was created. 
This file included the 3l^5 CaSes that were considered complete for analytical 
purposes and data for 311 jPistitutions.* It represents a merger of the final recipient- 
based data file, which inciiA<i^d best values and selected individual items from various 
data sources, instLtutLonaU^% for individual recipients and the institution as a whole, 
and nonresponse adjustment \^^eights for each recipient. (For a discussion of complete 
cases and nonresponse adjvi^trrierit see Chapter 7 of Volume 1.) 

All analysis was conc^tJCt^d using SAS, a comprehensive package for statistical 
analysis. SAS had also b^^*^ Us^d for the best value selection and the creation of all 
files used in analysis. Th^^^^tures of SAS, especially some new procedures included in 
the recently released 1982 *-^e^sior^, facilitated the accurate production of tables. 

6.^ CONTROL GROUPS 

One potential threat tCi thQ validity of the study was experimental bias--the 
impact of selection for p^f^i^ipation in the study on the findings. In order to assess 

*While 317 institutions (^participated, six of them were branch campuses which 
maintained no separate fir/^f^tial aid offices. Therefore, the financial aid office for 
the central campus, whiter* ih ali cases was included in our sample, served for the 
branch as well. 




6-13 



lis 



the impact of experimental bias on the study two control groups were established. 
One, the Institutional Control Group (ICG), consisted of students in the sampled 
institutions who were not selected for the study. Student aid files for these students, 
whose identity was not known to the institutions, were reviewed during the spring data 
coliection. These ICG participants were not interviewed, so comparisons could be 
made only on items related to institutional error. 

A second control group consisted of students whose records were randomly 
selected from the Computed Applicant Records of the Student Aid Report. Students 
selected for this control group, which involved computer tape data review only, were 
from both sampled and nonsampled institutions. 

The use of these two control groups to assess experimental bias is detailed in 
Chapter 7 of Volume 1, along with the -f*levant findings. Data for experimental bias 
assessment was analyzed using both SAS and SPSS, another statistical package. 




SURVEY RESPONSE RATES 



As in every voluntary survey, it was not possible to gain the cooperation of every 
institution to permit the drawing of a sample of its students or of every student and 
parent sampled at cooperating institutions. Nevertheless, we did achieve high survey 
response rates at both the institutional and individual levels. 

7 J INSTITUTIONAL SURVEY RESPONSE RATES 

The institutional sample initially drawn included 322 schools and colleges. For 
various reasons, five could not or would not cooperate with the survey. Therefore, the 
institutional response rate was 9rj.^ percent. 

The first occasion for institutional refusal occurred during the telephone calling 
to schedule the fall round of visits. Telephone schedulers who encountered refusals 
referred them to their Westat supervisor, who could refer them to Advanced 
Technology's institutional data collection manager, who was at the Westat telephone 
center to make rescheduling decisions. The data collection manager converted several 
refusals, including some based on an incorrect interpretation of the Buckley Amend- 
ment. Some refusals were referred to the project officer for resolution. The five 
u/timate refusals were made for the following reasons: 

• At one public four-year institution, a fire in the administration building had 
destroyed the financial aid records. 

• At two proprietary institutions, all financial aid matters were handled by a 
consultant who was abusively hostile, as he had been when two of his 
clients were included in the Stage One sample. 

• At one proprietary institution, the financial aid director and the director of 
the school were parties to litigation under trial and could not be available 
for the fall visit. They also refused to cooperate with the spring visits. 
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• At one proprietary institution, a new financial aid director was alrec.dy in 
the midst of a set of audits by accrediting organizations and other 
government agencies when we first contacted her. She also refused 
cooperation when recontacted in the spring. 

7.2 STUDENT AND PARENT SURVEY RESPONSE 

The number of completed cases was determined not only by the proportion of 
students and paients eligible for the study who cooperated by granting an interview, 
but also by what proportion of the initial sample we eventually determined not to have 
been part of the population being studied. We present the overall response rate first 
and then explain the problem of sample loss. 

7.2.1 Survey Response Rate 

The response rates presented in this section represent "final" disposition 
categories determined by i:he coders when the questionnaires were edited and coded, 
after receipt at Westat*s home office. In some cases, the disposition of an apparently 
completed questionnaire was changed as a result of additional work done at the home 
office after the questionnaire had been sent in as a completed instrument. For 
example, if a completed student questionnaire showed evidence that the student filed 
a "Special Condition Application" in applying for a Pell Grant, the financial aid 
administrator (FAA) at the student's institution was telephoned. If the FAA indicated 
that the student did file a Special Condition Application, the status of the 
questionnaire was changed from "complete" to "sample loss," because special condition 
filers should not have been in the sample. If the parent of that student also completed 
an interview, the parent's questionnaire was also changed from "complete" to "sample 
loss." 

Calculating the response rates from the clean data file represents a very 
conservative approach to specifying survey response statistics. Since the response 
rates cited in this section summarize the status of each questionnaire as it was edited 
by the home office coder, they slightly understate the response rates when compared 
to the operational response rates calculated during the field period and reported 
weekly to the Department of Education. 
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The dispositions of all the sampled students and parents are shown in Table 7-1. 
The percentages in the table represent the proportion of completed interviews of those 
eligible, sampled students and parents where an interview was possible. Ineligible 
sample members, principally students (and therefore their parents) who never actually 
received a Pell Grant or who filed a Special Condition Application, and sampled 
individuals who were dead or out of the country were deleted from the initial sample 
used as the denominator in the percentages. A separate analysif; of this sample loss 
category has been included in the report on findings as a part of the analysis of non- 
response. 

To satisfy the analytic needs of the Pell Grant QC study, both a parent and a 
student questionnaire must be completed for dependent student grants, while a student 
' questionnaire alone is sufficient to verify an independent student grant. The last row 
cf Table 7-1 reflects a combined response rate for pairs of dependent students and 
parents and for single independent students. 

7.2.2 Sample Loss 

The initial sample of ^,109 students was drawn early in the school year from two 
groups of students: those who were already receiving Pell Grants, and those who had 
been certified as eligible pending successful validation by the financial aid offices at 
their schools. We made efforts to identify and remove from the sampling frame 
students who did not receive a grant or who had used a Special Condition Application 
when they applied. It was, however, often impossible to identify these ineligible 
sampled students, particularly among the group of students pending validation. Table 
7-2 shows the number of student and parent cases and dispositions that make up the 
sample loss category. Because it includes both students and parents, the survey 
sample loss count is greater than the case sample loss count of ^22 reported in 
Chapter 7 of Volume 1. 

The sampled-in«error category represents students who were sampled in the fall 
as recipients or recipients pending validation, but who never actually received a grant, 
or who never actually attended the institution where they were sampled. If students 
were found to be in any of the sample loss categories, their parents were also included 
in that category, and both were removed from the data file. 
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PRELIMINAl^V STUDENT/PARENT SURVEY RESPONSE RATES 



Grant Type and Form 
of Questionnaire 



in S^fliple 



Dependent Student Grants: 

Student 
Parent 

Paired Student and Parent* 



Number 
Ineligible** 



198 
211 



Net 
Sample 



Number 
Completed 



2, 190 
2,177 
f ,332 



2,136 
2,09'f 
f ,lfO 



Pet cent 
Completed 



97.5 
96.2 
95.6 



Independent Student Grants: 

Student 
Parent 

Paired Student and Parent* 



lf5 
355 



l,5f5 
1,335 
2,638 



i,ffO 
967 
1,610 



93.2 
72A 
61.0 



Dependency Status Not Determined^ 



52 



59 



Dependent and Independent Grants: 

Paired Student and Parent 
Paired Student and Parent for 
dependent student grants and 
student only for independent 
student grants 



*Count of student and parent questionnaire ^r»ot a count of cases) 
**Sample loss 



6,970 



5,879 



5,750 



5,580 



82.5 



95.0 



ERIC 
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Although the sample loss represents a reduction in the overall sample size for 
the survey, it is well within acceptable limits. Since the number of 
completed interviews exceeded the target, the precision of the sample estimates is 
not seriously affected by the reduction. 



TABLE 7-2 

DISPOSITION OF STUDENT AND PARENT 
SURVEY SAMPLE LOSS CASES 





Number in 


Percent of 


Disposition Category 


Category 


Sample Loss 


Sampled in Error 






Out of Country 


161 


16.6% 


Deceased 


121 


12.5% 


Special Condition Filer 


221 


22.0% 


Total 


968 


100.0% 



7.2.3 Response Rate for Analytical Purposes 

The response rates presented in this chapter reflect the success of the efforts to 
interview students and their parents. Data analysis, however, requires not oniy that an 
interview be conducted but that all sections of the interview be complete and all other 
records be available. Thus, some cases that were considered complete for survey 
purposes have been recoded as incomplete for analytical purposes, reducing the survey 
response rate of almost 95 percent to a useful response rate of over 86 percent. This 
reduction was primarily accounted for by two reasons. 

• Parents of independent students were requested to complete only part of 
the interview, with income and asset information omitted. Because their 
participation was completely voluntary and these data would not be needed 
in most cases, this information was excluded to facilitate cooperation and 
protect their privacy. If, however, the student was later deemed to be 
dependent, the omitted parent information became essential for a case to 
be considered complete for analysis. In some instances, it was possible to 
obtain this additional information from parents; in most (116 cases) it was 
not. This reduced the functional response rate. 



123 



• For some recipients we could not obtain their Computed Applicant Record 
from the Student Aid Report, without which error could not be determined. 
The reason for this was a failure to match the Social Security number (SSN) 
on our records (which were verified during the interview) with either the 
original SSN or the current SSN on the SAR file. This occurred in 72 cases. 
Some of these failures to match were due to a damaged tape (the tape 
from the central processor was received with unreadable data on about one 
half of one percent of the blocks, which would be 16 cases if evenly 
distributed), the remainder did not match for unknown reasons. 

Additional details on the functional response rate are given in Chapter 7 of Volume 1. 

7.3 VARIANCE ESTIMATES 



A critical part of the findings in Volume 1 are program-wide estimates of error 
in the population of Pell recipients. These estimates, of course, may vary from the 
actual population figures to the extent that the sample differs from the population. A 
series of ^2 tables presents selected estimates, standard errors of the estimates, and 
coefficients of variation. 



7.3.1 Variance Estimation by the Method of Balanced Repeated Replication 



The sampling error of an estimate, based on any sample design using any 
estimation procedure, no matter how complex, may be estimated by the method of 
jeplications. Theoretically, this method is equivalent to the idea that the sample 
selection, collection of data, and est. mation procedures are independently carried 
through (replicated) several times. In practice, random 50 percent subsets of the 
survey results are selected and estimates formed from each. The dispersion of the 
resulting estimates can be used to measure the variance of the full sample. 



The method of replications has special advantages in reducing the complexity of 
variance computations. Another benefit is that it may be applied to compute sampling 
errors for higher-order statistics without the need for new variance expressions. 



The method consists of three steps: 



• Assemble data for the sample units that make up each of the replicates. 
This is equivalent to making a copy of the sample data for the units in each 
of the subsamples of the full sample. 
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• Perform the estimation procedure on each of the replicates. The same 
estimation procedure, prepared for the full sample, is applied separately to 
each of the replicates. 

• Calculate the dispersion of the resulting estimates among the replicates to 
estimate the variance of the full sample; a relatively simple computation 
formula is used that does not depend on the form of the estimate for which 
the variance is to be approximated. 

Definition of the Replicates. Each of the half-sample replicates prepared for 
variance estimation must satisfy two criteria: 

• The replicate must comprise a sample approximately one-half the size of 
the full sample. 

• The selection of the half-sample must observe the same sampling principles 
as the full sample. 

For the replicates defined for the Pell C-rant design, these criteria are satisfied 
by selecting half-samples of the units designated in the first stage of sampling in the 
full sample. For nonself-representing institutions, a replicate comprises all students 
selected in half of the clusters; in self-representing institutitons a replicate comprises 
half of the students selected at the school. Half of the Alternative Disbursement 
System (ADS) students were selected in each half-sample. 

Each of the 3^ certainty institutions is treated as a stratum. Half of the 
students in each institution were assigned half-sample code 1 and half , were assigned 
half-sample code 2. All ADS students were placed in stratum 35 with half assigned a 
code of 1 and half assigned a code of 2. 

Students in noncertainty institutions were assigned to a stratum and half-sample 
based on the geographic cluster of the institution. Clusters were paired (in the order 
they appeared in the sampling frame) to form 36 strata. All students in a cluster were 
assigned the same half-sample code (1 or 2). 

Half-samples of the full sample were defined by randomly selecting one or the 
other half-sample code from each of the 71 pairs; the number of different half-sample 
replicates possible by this method would equal 2 raised to the 71st power (abouth 2,361 
billion). McCarthy has shown that the variance can be estimated with equivalent 
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reliability from only a small number of orthogonal replicates.* For the Pell Grant 
program, the number of orthogonal replicates ne ;ded is the smallest multiple of four 
equal to or greater than the number of pairs. Wir^ 71 pairs, the number of replicates 
needed is 72, 

Variance Calculations. For every estimate X calculated from the total sample, 
let ^\ be the estimate calculated from the ith half-sample, i = 1, 2, 72. The 
variance estimate for X is then 

1 72 

Var (X) = — ~ 2 (x: - X)2. 

This formula has been used to compute the^ standard errors (square root of the 
variance) for each statistic presented in Tables 7-3 through 7-^^. 

7.3,2 Estimated Sampling Errors 

In this section we present estimated sampling errors developed using the methods 
described in the previous section. For each statistic we present the estimate itself, 
th^ standard error of the estimate, and the coefficient of variation (standard error of 
the estimate divided by the estimate). 

Table 7-3 presents the standard errors for program-wide dollar estimates of 
errc?r. The coefficients of variation for student and case error are always less than 12 
percent, and the coefficients are often as low as 6 percenc. Estimated standard errors 
for number of cases with payment or eligibility errors are presented in Table 7-^. The 
standard errors for estimated numbers of payment errors are at or below 8 percent. 
Occurrence of eligibility errors, except no Financial Aid Transcript, was quite rare. 
Therefore, the coefficients of variation will appear to be high. However, the standard 
err(;rs are low enough so that reasonably tight confidence intervals exist. 



U.S. Department of Health, Education and Welfare, National Center for Health 
Statistics. Replication; An Approach to the Analysis of Data from Complex Surveys 
by Philip 3- McCarthy, Series 2, No. 1^, Washington, D.C., Government Printing 
Ofiice. 
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TABLE 7-3 



ESTIMATED SAMPLING ERRORS: 
SELECTED PROGRAM-WIDE DOLLAR ESTIMATES 



Statistic 



Overaward 
Institution 
Student 
Case 



Estimate 
($ Millions) 



206.36 
265.25 



Standcird 
Error 
($ MilUons) 



26.68 
13.95 
29.00 



Coefficient 
of Variation 



0.13 
0.05 
U.06 



Underaward 
Institution 
Student 
Case 



lO'f.25 
5't.33 
136.20 



18.79 
5.67 
15.91 



0.18 
0.10 
0.12 



Net Error 

Institution 102.11 3'f.21 0.3'f 

Student 210.92 15.52 0.07 

Case 316.22 35.99 0.11 



Absolute Error 

Institution 310.62 30.98 0.10 

Student 319.58 l'f.58 0.05 

Case 588.63 29.88 0.05 



ERIC 
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TABLE 



ESTIMATED SAMPLING ERRORS: 
SELECTED NUMBER OF RECIPIENTS WITH ERROR 



Statistic 


Estimate 
(Thousands) 


Standard 
Error 
(Thousands) 


Coefficient 
of Variation 


Institutional Overaward 


388.05 


28.23 


0.07 


Institutional Underaward 


«7.20 


33.17 


0.08 


Student Overaward 


758.07 


30.08 


O.O^f 


Student Underaward 


218.81 


OA7 


0.06 


Case Overaward 


1,022.72 


31.90 • 


0.03 


Case Underaward 


525.29 


2^^.71 


0.05 



ELIGIBILITY ERRORS 



Unsatisfactory Academic 
Progress 

Less Than Half Time 

Insufficient Program Length 

Nondegree Program 

No Statement of Educational 
Purpose 

No Financial Aid Transcript 

Loan Default 

Invalid SAR 

Not Parent School 



9.22 

2.98 
0.76 
11.89 
9.38 

82.21 
0.70 
9.92 
5.10 



3.68 

l.i^S 
0.73 
1.72 
3.if5 

12.1^9 
0.66 
2.92 
2.16 



0.50 
0.95 
0.1ft 
0.37 

0.15 
0.96 
0.29 
0.^2 



ERIC 
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Standard errors for mean paynnent errors are presented in Table 7-5. These 
standard errors are generally less than $25; thus, 95 percent confidence intervals 
would generally have a width of less th^ri $100. 

The remaining tables provide sarripling errors for the various estimates for the 
following categories: 

Type and Control of Institution 
Grant Type (Dependency St^t^js) 
Validation Status 

Tables 7-6 to 7-17 present estimates f^r the various program-wide estimates of error. 
Tables 7-18 to 7-23 show estimates for t^^ number of recipients with errors. Tables 7- 
2^ to 7-35 contain sampling errors for i^iean payment errors; and, finally. Tables 7-36 
to 7-^^ show the estimated number of recipients with eligibility errors. 
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TABLE 7-5 



ESTIMATED SAMPLING ERRORS: 
SELECTED MEAN DOLLAR ESTIMATES 



Statistic 



Mean Overaward 
Institution 
Student 
Case 



Estimate 

($) 



532 
350 
^^2 



Standard 
Error 
($) 



1^5 
11 
23 



Coefficient 
of Variation 



0.08 
0.03 
0.05 



Vlean Underaward 
Institution 
Student 
Case 



238 
2^8 
259 



33 
15 
22 



0.06 
0.09 



Mean Net Error 
Institution 
Student 
Case 



^0 
83 
125 



1^ 
6 
I't 



0.3^ 
0.07 
0.11 



Mean Absolute Error 
Institution 
Student 
Case 



123 
126 
233 



12 
6 
12 



0.10 
0.05 
0.05 



ERIC 
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TABLE 7-6 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
INSTITUTIONAL OVERAWARD 



Category 



Type and Control of Instizution 
Public 4 Year 
Private 4 Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 
($ Millions) 



90.75 
49.98 
25.15 
2.79 
20.69 
17.00 



Standard 
Error 
($ MilMons) 



21.60 
11.56 
3.82 
1.80 
15.22 
2.63 



Coefficient 
of Variation 



0.24 
0.23 
0.15 
0.65 
0.74 
0.15 



Grant Type 

Independent 123.27 21.81 0.18 

Dependent 83.09 11.16 0.13 

Validation Status 

Non-Validated 86.42 12.05 0.14 

Validated 119.94 18.78 0.16 

All Categories Combined 206.36 26.68 0.13 
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TABLE 7-7 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
STUDENT OVERAWARD 



Category 



Type and Control of Institution 
Public Year 
Private Yean. 
Public 2 Year 
Private 2 Year 
Proprie':ary 2 Year 
Proprietary Less Than 2 Year 



Estimate 
($ Millions) 



130.16 
63. 2^^ 
1^1.32 
6.06 
12.18 
10.76 



Standard 
Error 
($ Millions) 



13.29 
9.03 
5.76 
2.29 
5.53 
3.39 



Coefficient 
of Variation 



0.10 

o.u 

0.38 
0.^*5 
0.31 



Grant Type 
Independent 
Dependent 



10^^.09 
161.16 



10.86 
8.2^* 



0.10 
0.05 



Validation Status 
Non-Validated 
Validated 



91.^*3 
173.82 



6.78 
13.35 



0.07 
0.08 



All Categories Combined 



265.25 



13.95 



0.05 
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TABLE 7-8 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 

CASE OVERAWARD 



Category 



Type and Control of Institution 
Public ^ Year 
Private Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 
($ Millions) 



217.61 
107.87 
61.73 
8.51 
30.51 
26.20 



Standard 
Error 
($ Millions) 



26.8it 
16.82 
7.06 
3.00 
17 A2 
it. 30 



Coefficient 
of Variation 



0.12 
0.16 
0.11 
0.35 
0.57 
0.16 



Grant Type 
Independent 
Dependent 



217.76 
23it.66 



23.26 
12.55 



O.il 
0.05 



Validation Status 
Non-Validated 
Validated 



168. if it 
283.99 



15.29 
21.52 



0.09 
0.08 



All Categories Combined 



it52.it3 



29.00 



0.06 
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TABLE 7-9 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
INSTITUTIONAL UNDERAWARD 



Category 



Type and Control of Institution 
Public t^ Year 
Private Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 
($ Millions) 



25.82 
1^^.95 
: .98 
3.3^^ 
7.70 
12. 'fO 



Standard 
Error 
($ Millions) 



3.'f3 
'f.ll 
15.50 
2.53 
3.15 
6.33 



Coefficient 
of Variation 



0.13 
0.27 
0.39 
0.76 
OAl 
0.51 



Grant Type 
Independent 
Dependent 



48.81 

55A5 



12„i9 
7.95 



0.25 
O.l^f 



Validation Status 
Non-Validated 
Validated 



^^6.97 
57.28 



9.76 
9.51 



0.21 
0.17 



All Categories Combined 



10^^.25 



18.79 



0.18 



ERIC 
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TA^t^ 7-10 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
STUDENT Ur^DERAWARD 



Category 



Type and Control of Institution 
Public It Year 
Private 4 Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 
($Mmons) 



25.81 
15.62 
8.01 
1.87 
1.56 
1.30 



Standard 
Error 
($ Millions) 



^^.54 
2.97 
2.28 
1.06 
0.76 
0.67 



Coefficient 
of Variation 



0.18 
0.19 
0.28 
0. 56 
OAS, 
0.52 



Grant Type 
Independent 
Dependent 



11.86 



3.26 



0.27 
0.11 



Validation Status 
Non-Validated 
Validated 



17.52 
36.82 



3.28 
3.70 



0.19 
0.10 



All Categories Combined 



54.33 



5.67 



0.10 
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TABLE 7-11 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 

CASE UNDERAWARD 



CateRory 



Type and Control of Institution 
Public ^ Year 
Private it Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 
($ Millions) 



25.34 
^^2.33 
if. 87 
6.89 
12.20 



Standard 
Error 
($ Millions) 



5.51 

13.88 
3.00 
2.52 
6.07 



Coefficient 
of Variation 



0.12 
0.18 
0.33 
0.62 
0.36 
0.50 



Grant Type 
Independent 
Dependent 



50. 7^* 
85. ^^6 



9.79 
8.^*7 



0.19 
0.10 



Validation Status 
Non- Validated 
Validated 



55.22 
80.99 



2.93 
S.l^t 



0.16 
0.10 



All Categories Combined 



136.20 



15.91 



0.12 



ERIC 
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TABLE 7-1.2 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
NET INSTITUTIONAL ERROR 



Category 



Type and Control of Institution 
Public Year 
Private 4 Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 
($ Millions) 



64.94 
35.03 
-14.83 
-0.55 
12.99 
4.54 



Standcird 

Error 
($ Millions) 



21.42 
13.24 
16.34 

2.96 
14.93 

7.23 



Coefficient 
of Variation 



0.33 
0.38 
1.10 
5.36 
1.15 
1.59 



Grant Type 
Independent 
Dependent 



74.47 
27.64 



25.82 
14.74 



0.35 
0.53 



Validation Status 
Non-Validated 
Validated 



39.45 
62.66 



16.44 
21.73 



0.42 
0.35 



All Categories Combined 



102.11 



34.21 



0.34 



ERIC 
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TABLE 7-13 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
ABSOLUTE INSTITUTIONAL ERROR 



Category 



Type and Contro'l of Institution 
Public 'f Year 
Private 'f Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 
($ Millions) 



116.57 
6f .93 
65.13 
6.13 
28.39 
29A6 



Standard 
Error 
($ Millions) 



22.32 
11.21 
15.59 

3.26 
16.13 

6A6 



Coefficient 
of Variation 



0.19 
0.17 
0.21^ 
0.53 
0.57 
0.22 



Grant Type 
Independent 
Dependent 



172.08 
138. 5f 



2f .12 
12.57 



O.lf 
0.09 



Validation Status 
Non-Vaj;d'-'ted 
Validated 



133.39 
177.23 



If .51 
20.36 



0.11 
0.11 



All Categories Combined 



310.62 



30.98 



O.IO 



ERIC 
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TABLE 7-lit 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
NET STUDENT ERROR 



Category 



Type and Control of Institution 
Public If- Year 
Private Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 
($ MilUons) 



lOif.35 
if?. 62 
33.91 

if. 18 
10.62 

9A6 



Standard 
Error 
($ Millions) 



13.^^0 
8.70 

1.87 
5.00 
3. IS 



Coefficient 
of Variation 



0.13 
0.18 
0.16 

0A5 
0A7 
0.31* 



Grant Type 
Independent 
Dependent 



92.23 
118.68 



12.00 
9.01 



0.13 
0.08 



Validation Status 
Non-Validated 
Validated 



73.92 
137.00 



7.97 
13.93 



0.11 
0.10 



All Categories Combined 



210.92 



15.52 



0.07 
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TABLE 7-15 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
' ABSOLUTE STUDENT ERROR 



Category 



Type and Control of Institutiof 
Public H- Year 
Private 4 Year 
Pijblic 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 
($ Millions) 



155.98 
78.86 
it9.93 
7.93 
13.75 
12.06 



Standard 
Error 
($ Millions) 



U.66 
10.25 
6.89 
3.03 
6.11 
3.71 



Coefficient 
of Variation 



0.09 
0.13 
0.1^ 
0.38 

0.31 



Grant Type 
Independent 
Dependent 



115. 9^^ 
203.6^ 



10.63 
9.89 



0.09 
0.05 



Validation Status 
Ncn-Validated 
Validated 



108.95 
210.63 



7.07 
13.77 



0.06 
0.07 



Ai) Categories Combined 



319.63 



1^.58 



0.05 
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ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 

NET CASE ERROR 



Category 



Type and Control of Institution 
Public Year 
Private Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 
($ Millions) 



173.03 
S2.5if 

3.63 
23.61 
l^f.OO 



Standard 

Error 
($ Millions) 



25. 1 1 
17.72 

2.73 
16. 9^* 
7.2if 



Coefficient 
of Variation 



0.15 
0.21 
0.80 
0.75 
0.72 
0.52 



Grant Type 
Independent 
Dependent 



167.02 
1^^9.21 



25.19 
16.25 



0.15 
0.11 



Validation Status 
Non-Validated 
Validated 



113.22 
203.00 



19.26 
2if.5'f 



0.17 
0.12 



All Categories Combined 



316.22 



35.99 



0.11 
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TABLE 7-17 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
ABSOLUTE CASE ERROR 



Category 



Type and Control of Institution 
Public if Year 
Private ^ Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 
($ Millions) 



262.18 
133.21 

13.38 

37. 'f0 

38. 'f0 



Standard 
Error 
($ Millions) 



29.52 
17.15 
15.66 

5.35 
IS. 2^ 

7.6^ 



Coefficient 
of Variation 



0.11 
0.13 
0.15 

OAO 
OAS 
0.20 



Grant Type 
Independent 
Dependent 



268.51 
320.12 



25.29 
13.96 



0.09 
O.O'f 



Validation Status 
Non-Validated 
Validated 



223.65 
364.97 



15.99 
21.37 



0.07 
0.06 



All Categories Combined 



58S,63 



29.88 



0.05 
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TABLE 7-18 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
NUMBER OF RECIPIENTS VflTH INSTITUTIONAL OVERAWARD 



Category 



Type and Control of Institution 
Public Year 
Private Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 
(Thousands) 



170.97 
60. if 9 

103.69 
if. 69 
23.25 
2k. 97 



Standard 
Error 
(Thousands) 



25.09 
14.49 
14.10 

2.05 
15.77 

4.64 



Coefficient 
of Variation 



0.15 
0.24 
0.14 
0.44 
0.68 
0.19 



Grant Type 
Independent 
Dependent 



207.99 
180.06 



21.77 
14.81 



0.10 
0.08 



Validation Status 
Non-Validated 
Validated 



161.04 
227.01 



14.96 
19.76 



0.09 
0.09 



All Categories Combined 



38S.05 



28.23 



0.07 
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TABLE 7-19 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
NUMBER OF RECIPIENTS VHTH INSTITUTIONAL UNDERAWARD 



Category 



Type and Control of Institution 
Public Year 
Private Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



* Estimate 
(Thousands) 



156.^9 
36.66 

171.58 
17.^8 
19.09 
35.90 



Standard 
Error 
(Thousands) 



20. if 7 
7.70 

23.11 

11.35 
6.63 

10.88 



Coefficient 
of Variation 



0.13 
0.21 
0.13 
0.65 
0.35 
0.30 



Grant Type 
Independent 
Dependent 



198.58 
238.63 



21 AO 
17.69 



0.11 

0.07 



Validation Status 
Non-Validated 
Validated 



180.7^ 



17.37 
20.83 



0.10 
0.08 



All Categories Combined 



^37.20 



33.17 



0.08 



7-26 
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TABLE 7-20 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
NUMBER OF RECIPIENTS WITH STUDENT OVERAWARD 



Category 



Type and Control of Institution 
Public 4 Year 
Private 4 Year 
Pubiic 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 
(Thousands) 



365.56 
194.35 
122.95 
21.64 
19.73 
31.88 



Standard 
Error 
(Thousands) 



31.89 
28.88 
14.86 
8.89 
8.19 
7.65 



Coefficient 
of Variation 



0.09 
0.15 
0.12 
0.41 
0.41 
0.24 



.Grant Type 

Independent 
Dependent 



183.28 
574.78 



16.82 
23.28 



0.09 
0.04 



Validation Status 
Non-Validated 
Validated 



254.70 
503.36 



14.04 
28.37 



0.06 
0.06 



All Categories Combined 



758.07 



30.08 



0.04 



ERIC 



7-27 
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TABLE 7-21 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
NUMBER OF RECIPIENTS WITH STUDENT UNDERAWARD 



Category 



Type and Control of Institution 

Public Year 

Private ^- Year 

Public 2 Year 

Private 2 Year 
, Proprietary 2 Year 

Proprietary Less Than 2 Year 



Estimate 
(Thousands) 



100. 10 
62.29 
31.13 
8.03 
6.81 
SA3 



Standard 
Error 
(Thousands) 



1^^.35 
8. S3 
6.50 

3.18 



Coefficient 
of Variation 



C. lif 
0.21 
0.55 
0.36 
0.37 



Grant Type 
Independent 
Dependent 



■ W.33 
178.^*8 



6.33 
12.91 



0.16 
0.07 



Validation Status 
Non-Validated 
Validated 



6k. 72 



6.47 
10.91 



0.10 
0.07 



All Categories Combined 



218.81 



13.^*7 



0.06 



7-28 
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TABLE 7-22 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
NUMBER OF RECIPIENTS WITH CASE OVERAWARD 



Category 



Type and Control of Institution 
Public Year 
Private k Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 
(Thousands) 



225.38 
193.05 
2'4A5 
35.89 
'A 'A. 85 



Standard 
Error 
(Thousands) 



kk. 10 
31.08 
22.26 

9.11 
19.09 

6.it9 



Coefficient 
of Variation 



0.09 

o.m- 

0.12 
0.38 
0.53 



Grant Type 

* 

Independent 
Dependent 



350.98 
671. Z^A 



25.87 
21.92 



0.07 
0.03 



Validation Status 
Non-Validated 
Validated 



367. 0«A 
655.68 



21.76 
28.26 



0.06 



Ml Categories Combined 



1,022.72 



31.90 



0.03 



7-29 
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TABLE 7-23 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
NUMBER OF RECIPIENTS WITH CASE UNDERAWARD 



Category 



Type and Control of Institution 
Public k Year 
Private Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 
(Thousands) 



189.39 
8S.50 

171.97 
18.13 
16.05 



Standard 
Error 
(Thousands) 



22.08 
12.96 
21-02 
10.87 
5.38 
11.23 



Coefficient 
of Variation 



0.12 
0.15 
0.12 



0.60 
0.33 
0.27 



Grant Type 
Independent 
Dependent 



201.20 
32^^.09 



17.88 
17.60 



0.09 
0.05 



Validation Status 
Non- Validated 
Validated 



195.36 
329. 9^^ 



lif.Ol 
16.30 



0.07 
0.05 



All Categories Combined 



525.29 



2^^.71 



0.05 



7-30 

Us 



TABLE 7-2^ 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
MEAN INSTITUTIONAL OVERAWARD 



Category 



Type and Control of Institution 
Public ^ Year 
Private ^ Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 

($) 



531 
S26 
2*3 
595 
890 
681 



Standard 
Error 
($) 



80 
89 
32 
338 
251 
120 



Coefficient 
of Variation 



0.15 
0.11 
0.13 
0.57 
0.28 
0.18 



Grant Type 
Independent 
Dependent 



593 
461 



63 
1^2 



0.11 
0.09 



Validation Status 
Non-Validated 
Validated 



537 
528 



53 
55 



0.10 
0.10 



All Categories Combined 



532 



1^5 



O.OS 



7-31 
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TABLE 7-25 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
MEAN STUDENT OVERAWARD 



Category 



Type and Control of Institution 
Public ^ Year 
Private ^ Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 

($) 



356 
325 
3'H 
280 
617 
338 



Standard 
Error 
($) 



22 
32 
2tt 
175 
256 
78 



Coefficient 
of Variation 



0.06 
0.10 
0.07 
0.62 
O.'tl 
0.23 



Grant Type 
Independent 
Dependent 



568 
280 



25 
11 



O.Oif 
O.O^f 



Validation Status 
Non-Validated 
Validated 



359 
3'>5 



20 
15 



0.05 
O.Oif 



All Categories Combined 



350 



11 



0.03 



7-32 
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T/vBLE 7-26 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
MEAN UASE OVERAWARD 



Category 



Type and Control of Institution 
Public ^ Year 
Private ^ Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 

($) 



436 

320 
352 
850 
58^^ 



Standard 
Error 
($) 



33 

25 
107 

67 



Coefficient 
of Variation 



0.08 
0.09 
0.08 
0.30 
0.17 
0.11 



Grant Type 
Independent 
Dependent 



620 



16 



0.06 
0.05 



Validation Status 
Non-Validated 
Validated 



^^59 



31 
25 



0.07 
0.06 



All Categories Combined 



23 



0.05 



7-33 
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TABLE 7-27 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
MEAN INSTITUTIONAL UNDERAWARD 



Category 



Type and Control of Institution 
Public ^ Year 
Private ^ Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 

($) 



165 

233 
191 



Standard 
Error 
($) 



17 
61 
70 
59 
132 
109 



Coefficient 
of Variation 



0.10 
0.15 
0.30 
0.31 
0.33 
0.32 



Grant Type 
Independent 
Dependent 



2^*6 
232 



1^9 
26 



0.20 
0.11 



Validation Status 
Non-Validated 
Validated 



260 
223 



39 
31 



0.15 



All Categories Combined 



238 



33 



TABLE 7-28 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
MEAN STUDENT UNDERAWARD 



Category 



Type and Control of Institution 
Public ^ Year 
Private ^ Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 

($) 



258 
251 
257 
233 
229 
153 



Standard 
Error 
($) 



21 
29 

ISl 
It5 



Coefficient 
of Variation 



0.08 
0. 11 
0.16 
0.78 
OAl 
0.29 



Grant Type 
Independent 
Dependent 



29^^ 
238 



55 
15 



0.19 
0.06 



Validation Status 
Non-Validated 
Validated 



271 
239 



in 

13 



0.15 
0.05 



All Categories Combined 



16 



0.06 



7-35 
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TABLE 7-29 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
MEAN CASE UNDERAWARD 



Type and Control of Institution 
Public ^ Year 
Private ^ Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 

($) 



235 
286 

269 
296 



Standard 
Error 
($) 



17 
27 
60 
76 
150 
99 



Coefficient 
of Variation 



0.07 
0.09 
0.2tt 
0.28 
0.35 
0.33 



Grant Type 
Independent 
Dependent 



252 
26^^ 



36 
18 



0.14 
0.07 



Validation Status 
Non-Validated 
Validated 



293 
21^5 



3* 
19 



0.12 
0.08 



All Categories Combined 



259 



22 



0.09 



7-36 
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TABLE 7-30 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
MEAN NET INSTITUTIONAL ERROR 



Category 



Type and Control of Institution 
Public ^ Year 
Private ^ Year 
Public 2 Year 
Private 2 Vear 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estinnate 

($) 



26 
1* 
-6 
-0 

5 
2 



Standard 
Error 
($) 



8 
5 
6 
I 
6 
3 



Coefficient 
of Variation 



0.33 
0.38 
1.10 
5.36 
1.15 
1.59 



Grant Type 
Independent 
Dependent 



29 
11 



10 
6 



0.35 
0.53 



Validation Status 
Non-Validated 
Validated 



16 
25 



6 
9 



0.42 
0.35 



All Categories Combined 



*0 



1* 



0.34 



ERIC 



7-37 
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TABLE 7-31 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
MEAN ABSOLUTE INSTITUTIONAL ERROR 



Category 



Type and Control of Institution 
Public ^ Year 
Private ^ Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 

($) 



it6 
26 
26 
2 
11 
i2 



Standard 
Error 
($) 



9 
It 
6 
1 
6 
3 



Coefficient 
of Variation 



0.19 
0.17 
0.2't 
0.53 
0.57 
Q.?'> 



Grant Type 
Independent 
Dependent 



68 
55 



10 
5 



O.l^f 
0.09 



Validation Status 
Non-Validated 
Validated 



53 
70 



0.11 
0. 11 



All Categories Combined 



123 



12 



0. iO 



7-38 
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TABLE 7-32 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
MEAN NET STUDENT ERROR 



Category 



Type and Control of Institution 
Public ^ Year 
Private ^ Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 

($) 



19 
13 
2 



Standard 
Error 
($) 



5 
3 
2 
1 
2 
1 



Coefficient 
of Variation 



0.13 
0.18 
0.16 

QA7 
0.3if 



Grant Type 
Independent 
Dependent 



36 

k7 



5 
I* 



0.13 
0.08 



Validation Status 
Non-Validated 
Validated 



29 
5if 



3 
6 



0.11 
0.10 



All Categories Combined 



S3 



0.07 



7-39 
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TABLE 7-33 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
MEAN ABSOLUTE: STUDENT ERROR 



Category 



Type and Control of Institution 
Public ^ Year 
Private ^ Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 

($) 



62 
31 
20 
3 
5 
5 



Standard 
Error 
($) 



6 

3 

1 
2 
1 



Coefficient 
of Variation 



0.09 
0.13 
O.l'f 
0.38 

0.31 



Grant Type 
Independent 
Dependent 



46 
80 



if 



0.09 

0.05 



Validation Status 
Non- Validated 
Validated 



1^3 
83 



0.06 
0.07 



All Categories Combined 



126 



0.05 



7-W 
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TABLE 7-3* 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
MEAN NET CASE ERROR 



Category 



Type and Control of Institution 
Public * Year 
Private * Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 

($) 



6S 
33 
8 
1 
9 
6 



Standard 
Error 
($) 



10 
7 
6 
1 
7 
3 



Coefficient 
of Variation 



0.15 
0.21 
0.80 
0.7.5 
G.72 
0.52 



Grant Type 
Independent 
Dependent 



66 
59 



10 
6 



0.15 
0.11 



Validation Status 
Non-Validated 
Validated 



1^5 
80 



8 

10 



0.17 
0.12 



AirCategories Combined 



125 



I'f 



0.11 
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TABLE 7-35 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
MEAN ABSOLUTE CASE ERROR 



Category 



Type and Control of Institution 
Public ^ Year 
Private ^ Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estinnate 

($) 



53 
5 

15 
15 



Standard 
Error 
($) 



12 
7 
6 
2 
7 
3 



Coefficient 
of Variation 



0.11 
0.13 
0. 15 
O.W 

0.20 



Grant Type 
Independent 
Dependent 



106 
127 



10 
6 



0.09 
O.Oti- 



Validation Status 
Non-Validated 
Validated 



88 



6 
8 



0.07 
0.06 



All Categories Combined 



233 



12 



0.05 



7-1^2 
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TABLE 7-36 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
NUMBER OF RiiCIPIENTS WITH UNSATISFACTORY ACADEMIC PROGRESS ELIGIBILITY ERROR 



Standard Coefficient 

Category Estimate Error of Variation 

Type and Control of Institution 

Public ^ Year 6,2^6 3,396 0.5^ 

Private ^ Year 737 697 0.95 

Public 2 Year 2,235 1,551 0.69 

Private 2 Year 0 o ~ 

Proprietary 2 Year 0 0 

Proprietary Less Than 2 Year 0 0 

Grant Type 

Independent ^,777 2^623 0.55 

Dependent ^,^^1 1,7^8 0.39 

Validation Status 

Non-Validated 78^ 735 0.9^ 

Validated 8,^3^ 3,597 " 0.^3 

All Categories Combined 9,218 3,682 0.^0 



7-^3 



TABLE 7-37 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
NUMBER OF RECIPIENTS WITH LESS THAN HALF TIME ELIGIBILITY ERROR 



Standard Coefficient 

Category ^_ Estimate Error of Variation 

Type and Control of Institution 

Public ^ Year 1 , 66 953 0 . 65 

Private ^ Year 0 0 

Public 2 Year 0 0 — 

Private 2 Year 0 , o — 

Proprietary 2 Year 727 676 0.93 

Proprietary Less Than 2 Year 78^f '756 0.96 

Grant Type 

Independent l,^f66 953 0.65 

Dependent 1,511 1,077 0.71 

Validation Status 

Non-Validated 1,511 1,077 0.71 

Validated l,if66 953 0.65 

All Categories Combined 2,977 1,^8^ 0.50 
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TABLE 7-38 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
NUMBER OF RECIPIENTS WITH INSUFFICIENT PROGRAM LENGTH ELIGIBILITY ERROR 



Standard Coefficient 

Category Estimate Error of Variation 

Type and Control of Institution 

Public ^ Year 0 0 

Private ^ Year 0 0 — 

Public 2 Year 0 0 ~ 

Private 2 Year 0 0 — 

Proprietary 2 Year 0 0 

Proprietary Less Than 2 Year 765 726 0.95 

Grant Type 

Independent 0 0 

Dependent 765 726 0.95 



Validation Status 

Non-Validated 765 726 0.95 

Validated 0 0 

All Categories Combined 765 726 0.95 



7-^5 
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TABLE 7-39 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
NUMBER OF RECIPIENTS WITH NONDEGREE PROGRAM ELIGIBILITY ERROR 



Standard Coefficient 

Category Estimate Error of Variation 

Type and Control of Institution 

Public 4 Year 0 0 

Private 4 Year 708 675 ,0.95 

Public 2 Year 0 0 

Private 2 Year 0 0 — 

Proprietary 2 Year 0 0 

Proprietary Less Than 2 Year 11,184 1,672 0.15 

Grant Type 

Independent 8,218 4,890 0.59 

Dependent 3,674 3,472 0.95 

Validation Status 

Non-Validated 9,690 858 0.09 

Validated 2,202 1,181 0.54 

All Categories Combined 11,892 1,717 0.14 
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TABLE 7-40 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
NUMBER OF RECIPIENTS WITH NO STATEMENT OF EDUCATiONAL PURPOSE ELIGIBILITY ERPOl 



Category 



Type and Control of Institution 
Public Year 
Private Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Leas Than 2 Year 



Estimate 

3,357 
2,186 
2,957 
0 

0 



Standai d 
Error 



1,876 
1, 121 
2,226 
0 

8S3 

0 



Coefficient 
of Variati<./n 



0.56 
0.51 
0.75 



1.00 



Grant Type 
Independent 
Dependent 



5,676 
3,708 



2,750 
1,568 



O.itS 
0A2 



Validation Status 
Non-Validated 
Validated 



2,503 
6,881 



l,'f'f2 
2,69f 



0.58 
0.39 



All Categories Combined 



9,38^^ 



3,'f5. 



0.37 



ERIC 



7-^7 
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TABLE 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
NUMBER OF RECIPIENTS WITH NO FINANCIAL AID TRANSCRIPT ELIGIBILITY ERROR 



Category 



Type and Control of Institution 
Public 4 Year 
Private Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 



'to, 779 
22, 03.'' 
9,525 
2,186 
6,163 
1,519 



Standard 
Error 



11,266 
6,025 
1^,1^2,5 

3,1^07 
972 



Coefficient 
of Variation 



0.28 
0.27 
OA? 
0.71 
0.55 



0.6^* 



Grant Type 
Independent 
Dependent 



't6,59't 
35,617 



5,018 
5,92^* 



0. 19 
0.1. 



Validation Status 
Non-Validated 
Validated 



33,1)0 
^^9,101 



5,2^*6 
8,952 



0.16 
0.18 



All Categories Connbined 



82,211 



12,^*87 



0.15 



ERIC 



7-^*8 



166 



TABLE 7-'t2 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
NUMBER OF RECIPIENTS WITH LOAN DEFAULT ELIGIBILITY ERROR 



Standard Coefficient 

CateRory Estimate Error of Variation 

Type and Control of Institution 

Public 4 Year 695 66* 0.96 

Private 4 Year 0 0 — 

Public 2 Year 0 0 — 

Private 2 Year 0 0 — 

Proprietary 2 Year 0 » 0 

Proprietary Less Than 2 Year 0 0 

Grant Type 

Independent 695 66^^ 0,96 

Dependent 0 0 

Validation Status 

Non-Validated 0 0 — 

Validated 695 66^ 0.96 

All Categories Combined 695 664 0,96 
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^ 187 
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TABLE 7-tt3 



ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
NUMBER OF RECIPIENTS WITH INVALID SAR ELIGIBILITY ERROR 



Category 



Type and Control of Institution 
Public ^ Year 
Private ^- Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 

if, 380 

2,566 
0 

73* 
780 



Standard 
Error 

1,998 
981 
1,873 
0 

700 

765 



Coefficient 
of Variation 

OAG 
0.67 
0.73 

0.95 
0.98 



Grant Type 
Independent 
Dependent 



4,086 
5,837 



2,083 
2,«f«f5 



0.51 

0A2 



Validation Status 
Non-Validated 
Validated 



2,619 
7,30«f 



1,950 
2,665 



0.36 



All Categories Combined 



9,923 



2,91S 



0.29 



7-50 
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TABLE 

ESTIMATED SAMPLING ERRORS BY SELECTED CATEGORIES: 
NUMBER OF RECIPIENTS WITH NOT PARENT SCHOOL ELIGIBILITY ERROR 



Category 



Type and Control of Institution 
Public 4 Year 
Private Year 
Public 2 Year 
Private 2 Year 
Proprietary 2 Year 
Proprietary Less Than 2 Year 



Estimate 

5,10(4- 
0 
0 
0 
0 
0 



Standard 
Error 



2,158 
0 
0 
0 
0 
0 



Coefficient 
of Var lotion 



0Ji2 



Grant Type 
Independent 
Dependent 



1,^*66 
3,638 



81 

2,150 



0 . 59 



Validation Status 
Non-Validated 
Validated 



1,^*59 
3,645 



1 , 026 
1,241 



All Categories Combined 



5,104 



2,158 



0.42 



7-51 
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